Fun fact: Computer security has a famous 2014 paper on how dramatically different assumptions and practices must be when dealing with the most motivated attacker in the world, who is after -YOU-. This is, -literally-, known "Mossad vs not-Mossad." Note the phone replacement.

Most code retrieval benchmarks optimize for synthetic problems. But professional software engineers don't solve artificial puzzles--they navigate large, often messy codebases. Optimizing for synthetic benchmarks means climbing the wrong hill. Models that top artificial leaderboards consistently stumble in real-world applications.