Pinned Tweet
Real client briefs, judged by real designers. 3,591 votes later, designers ranked #1.
The Frontier Design Benchmark is live.
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
ok some back story of why we built this benchmark:
1. i love design. designers can benefit so much from actually useful ai models. most models will get there within a few months.
i want to know which model is best to use for design, that’s why we are creating a gallery so I (and you) can see myself (yourself)
2. i want to see models evaluated on realistic, complex design tasks, not just landing pages. in frontier design, we now have handful of web app/mobile app prompts, but we will expand to more types of product/non-product design
4. i don’t believe “taste” is such a mysterious thing that models cannot learn. we curated a list of rubrics based on practical design principles to use as evaluation criteria. our analysis found that these rubrics’ grading is highly correlated with expert preference on model outputs. we will do some post training experiments on these rubrics
5. ai models have so many slop patterns! our team curated a list of slop patterns (with some serious data analysis pipeline to back it up). models improve and slop patterns change, so we will keep iterating on these
a really interesting part: we were able to identify some model specific slop patterns: claude likes to use right arrows ➡️ while gpt tends to use the super narrow top right arrow ↗️. stay tuned and we will share as we go!
6. i tried so many design skills and mcps but not sure which ones actually work. we will be working on this, and there’s gonna be some trajectory inspection involved
if you have any topic you are interested in please lmk!
design will look very different very soon. we do this for nothing but the love of design 💖
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
Yep. This is cool. Solid benchmark.
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
I’ve been a designer for almost 10 years, and honestly, AI has completely changed the way I approach creative work.
But here's something I've realized: generating beautiful design is one thing. Making everything work together is a completely different skill.
My process these days?
I start with @claudeai to explore brand strategy, positioning, and messaging.
Then I jump into @floraai to experiment with visual directions and creative concepts.
From there, I bring everything into @framer and @Lovable to build quick, interactive prototypes.
And this is my favorite part: watching the strategy, visuals, storytelling, and interactions dance together.
Suddenly, we're not looking at a collection of pretty AI-generated images. We're looking at something that feels like a real brand, a real website, a real experience.
Then comes the most important part: stepping back, questioning everything, testing ideas with clients, and deciding what actually works.
That's where human creativity, taste, and creative direction make all the difference.
And I think that's also how I recognize AI-only design.
It can look incredibly polished, but sometimes there's no clear intention connecting the pieces.
That's why I LOVE what @Contra Labs is doing with the new Frontier Design Benchmark.
Their findings show that designers working with AI still outperform AI models working alone.
AI gives us the freedom to explore a thousand directions.
Our job as designers is to know which one is worth pursuing.
contralabs.com/frontierdesig… – at Manhattan, NY
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
if we don’t have people with taste (designers) making, training, and deciding what gets used, we will be totally lost
love to see @contra’s dedication to this
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
Breaking🚨: There is a new model "Gestalt" that is beating everything, including the human golden designs 👀
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Readers added context they thought people might want to know
This announcement comes from the creator of the referenced benchmark at Contra Labs. "Gestalt (stealth)" is an anonymous model included in their own leaderboard, with no independent verification or public details available.
contralabs.com/frontierdesign
contralabs.com/research/front…
Contra Labs retweeted
As models continue to hill climb on design outputs, human will continue to raise the ceiling.
We wanted a better way to measure this that's not just crowdsourced opinions. We took real world, marketplace briefs and tested them against humans and models. Outputs were measured by real professional designer preference, a validated design critique rubric (grounded in how real designers, design), and AI slop detectors.
The results are both visually appealing and mentally stimulating (check it out -> contralabs.com/frontierdesig…)
The takeaway? Humans still beat models at the frontier (with a plot twist, see thread).
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
One of the biggest challenges in AI benchmarking is measuring what actually matters.
Design is a perfect example.
A model can generate a beautiful interface, but can it understand a real client brief and execute the vision?
@contralabs is asking the right questions with its Frontier Design Benchmark.
This is how AI capabilities should be evaluated.
🤝 Paid partnership
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
I spot AI-made design by over-usage of eyebrow labels, subtext helpers, and gradients.
@contra's benchmark found them on 93% of AI pages vs 11% of human ones.
And I like them, so this isn't a dunk. But it's just an observation that is now backed by data.
We need more design benchmarking, and this is a great start.
The AI industry has a benchmarking problem.
We're letting random people on the internet decide which models are best
even for capabilities like design...
So we built something different.
Introducing The Frontier Design Benchmark from @contralabs
contralabs.com/frontierdesig…
Contra Labs retweeted
Frontier visual AI shouldn't require billions in compute and Internet-scale data.
Our first model was trained from scratch by a small team, using a fraction of the resources typically required.
Check out Contra Labs' evaluation against 8 leading image models for first results. 🍑
We compared @genpeachAI v1 Preview against 8 image models on 100 human-centered briefs, judged blind by 53 professional creatives. It won 71% of votes.
The set: GPT-Image-2.5 Sunburst, Nano Banana 2, Muse Image, Recraft V4.1, Luma Uni-1, Flux 3, Ideogram 4.5 and Krea 2 Medium.
Contra Labs retweeted
Building frontier visual AI shouldn't require billions $ in compute and Internet-scale data.
A year after leaving Meta, I’m excited to share the first independent evaluation by Contra Labs of what we've built at @genpeachAI.
Our first foundation image model. Trained from scratch.
Ranked top-1 on human-centered image generation across all axes! Outperformed the large frontier models.
Small team. A fraction of the resources. More to come! 🚀
@contralabs eval report: contralabs.com/research/genp…
We compared @genpeachAI v1 Preview against 8 image models on 100 human-centered briefs, judged blind by 53 professional creatives. It won 71% of votes.
The set: GPT-Image-2.5 Sunburst, Nano Banana 2, Muse Image, Recraft V4.1, Luma Uni-1, Flux 3, Ideogram 4.5 and Krea 2 Medium.
GenPeach led on all four dimensions. Its widest lead was art direction: no model won more than 38% of votes against it. Nano Banana 2 came closest on human realism. Muse Image and GPT-Image-2.5 Sunburst came closest on brief adherence.
1
4
165
This study was independently conducted by Contra Labs using blind evaluation methodology, and was commissioned by GenPeach.
All nine models were evaluated under the same generation protocol, with exactly one image per prompt and no retries. Contra Labs conducted the blind evaluation using outputs from a fixed GenPeach model checkpoint, provided by GenPeach, and outputs generated through the other eight models' public APIs.
Read the full study with methodology and expert quotes: contralabs.com/research/genp…
4
141
Contra Labs retweeted
As startups are spending a lot more on storytelling and putting significant ad dollars behind campaigns
I rarely see startups tracking 2 very important metrics:
Brand lift: growth in awareness, perception, or intent resulting from marketing
Brand mindshare: how prominently your startup comes to mind within a category
@contralabs human creativity report looked at tools by post mindshare, which I really like as an ongoing metric to track.
Contra Labs retweeted
Hey X, this one is special,
@contralabs looking for a Head of Creative to push the boundaries of what's possible.
This is an opportunity to create things that have never been made before setting the bar for frontier AI capabilities.
You will collaborate directly with me and our research team to create one-of-one digital experiences that demonstrate the true capabilities of frontier AI models.
Location: SF or NY.
Comp: $180K - $250K + equity
DM me for job listing or if you have a referral!
PS. you will help us make creative assets, sites and repots like the ones below
AI is making creativity more valuable than ever before and..
we have 3 years of data to prove it.
Introducing the Creative Intelligence 2026 report from @contralabs.
Full report: c87.to/creative-intelligence…
Contra Labs retweeted
hey x, its that time again
@contralabs is hiring Researchers in SF/NY that are interested in making AI supercharge (not replace) creativity.
You will collaborate on cutting edge projects directly with frontier labs and researchers that impact the tools and models creatives use every single day.
If this is you, please DM or tag someone who could be interested!
hey x,
@contralabs_ai is hiring Researchers in SF/NY that are interested in making AI better for creativity.
You will collaborate directly with the frontier labs on cutting edge projects that directly impact the tools creatives use every single day
tag someone who could be interested!
Contra Labs retweeted
Big things are happening at @contralabs , and now's a great time to get in early ✨ We're growing and looking for more Researchers + Strategic Projects Leads to join us in SF and NY. Come build with us!


















