As new models emerge and benchmark scores keep improving, there's a temptation to reach for the flashy new model. But performance may degrade on bespoke tasks, and some models might surprise you if you give them a chance. Private benchmarks are key to these discoveries. I prompted a range of models to produce research designs sitting at the frontier of questions in political psychology and behavior. Through blind ratings of outputs, I selected those that mirrored the design choices I would personally make and could serve as research design co-partners. Contrary to my expectations, models like Opus 4.8 that I routinely use for coding produced overly elaborate designs. Smaller proprietary models, and the much-hyped GLM-5.2, generated viable designs. On tasks where I have ground-truth values, smaller open-source models like Qwen3.6 outperformed some of the proprietary models we might reach for when classifying text or generating stimuli. Cutting through the sea of AI hype we encounter on this platform and elsewhere can be exhausting. Building private benchmarks can ground our judgments. There are many ways to do this. @HannoHilbig and @dasanaike have been testing model performance on tasks like text classification and OCR, where you might already have gold-standard classifications or ratings. For taste-based judgments, you can ask a model to generate a research design, measurement approach, or recruitment strategy; select the outputs that meet your standards; and build a rubric that can be automatically applied to future models. @ahall_research argues that we should all be developing private benchmarks, and he's right. The field moves too fast for any single leaderboard to tell you what works for your problem. Only you can do that.