MineBench (@minebench_ai) header
MineBench (@minebench_ai) profile picture

An open-source, 3D spatial-reasoning benchmark for evaluating LLMs Github Repository: https://teapawt.org/t.co/mz6gm3s9mC

Joined April 2026
MineBench.ai 4.0 is live – our biggest release yet! Generations with Gemini 3.7 Flash for free for a limited time. No API key required! Make anything you want, publish it to the new Gallery, and vote on the prompts you want to see added to the official benchmark :) Also new: - Generate from any prompt, save your builds, and publish them to the new Gallery - Completely redesigned MineBench + a much faster Arena - Accounts + personal model rankings - Global rankings now use Bradley–Terry + 95% confidence intervals (Oh and MineBench is now on the App Store :) Full release notes: github.com/Ammaar-Alam/mineb…
8
1
1
18
13,603
MineBench retweeted
Gemini 4 Hydrogen ended up with a very interesting anamorphic projection on one it's minebench.ai builds
1
1
7
348
Claude Fable 5.2 is live on MineBench.ai, with significant efficiency jumps over Fable 5.1! - Total Cost (15 builds): $147.55 → $38.08 (-74%) - Avg Cost / Build: $9.84 → $2.54 - Avg Inference Time: 40m 11s → 12m 55s (3.1x faster) - Failed Attempts / Retries: 8 → 0 (15/15 one-shotted) The massive reduction in cost is a result of Fable 5.2's lower API pricing, but also the model requiring zero retries in our harness (15 total generations vs. Fable 5.1's 23).
2
1
11
502
Gemini 4 Carbon is now on MineBench.ai :) - Total Cost: $11.65 - Average Inference Time: 16m 0.9s Gemini 4 Argon (for reference): - Total Cost: $9.12 - Average Inference Time: 6m 15.1s Though its ratings are yet to stablize, it currently overtakes Argon for 2nd place on our leaderboards!
1
15
603
Models have gotten pretty darn good at making voxel phoenixes Data from MineBench
5
2
39
3,875
small iOS update 📖
1
3
189
MineBench retweeted
You can generate beautiful 3D builds on MineBench.ai for free (for Gemini 3.7/3.8 Flash), btw.
2
8
361
Gemini 4 Argon enters at the top spot on MineBench.ai :) - Total Cost: $9.12 - Average Inference Time: 6m 15.1s Its rating has yet to stabilize, but it currently pushes GPT-6.1 Sol Pro and GPT-6 Astra Pro off both the cost and speed Pareto frontiers by a significant margin. Also added to MineBench recently: GPT-6.1 Sol Pro, Claude Sonnet 5.5, and Muse Wonder! We’ve been selecting more community prompts for the official benchmark as well! Submit something you’d like to see the models try, and vote in the Arena for the builds you think best match the prompt!
7
14
8,955
We haven't posted many announcements as of late, but there's been a lot of work behind the scenes! We've continued making improvements and optimizations across both the site and iOS app, which now has a few hundred total downloads :D On top of adding more models and prompts, the majority of our work has been on the research and evaluation side, including things like defining what makes a good build, understanding where current frontier models still fall short, and figuring out how to measure those gaps more reliably. The prompts, builds, and human feedback that have accumulated on MineBench are becoming a big part of that work, alongside ongoing conversations with labs and researchers about where to take these evaluations next. We're also training a model as part of this work. We'll share more about everything we've been working on in the coming weeks :)
2
69
MineBench retweeted
DeepSeek v4.1 Flash vs. GPT-6.1 Sol
4
1
23
2,038
Yes, that photorealistic bust of Einstein is still entirely made up of individual Minecraft blocks! With the help of our sponsors, we've finally been able to add another batch of community prompts to MineBench.ai! The models are getting so good that we're also officially raising the bar for future benchmark prompts. We now accept prompts going up to a gridSize of 512³ instead of just 256³ (8× the resolution!), along with a block palette of 322 different block types instead of 25. Some thoughts: - GPT 6.1 Sol's generations are extremely close to GPT 6 Astra's, and at times, indistinguishable. Some may even look slightly better, though Astra still seems to maintain a slight edge in overall accuracy and realism, especially with its proportions. - Opus 5.5 seems like a huge step up from Opus 5. Unlike its predecessor, it's much less prone to over-engineering and adding excess details. It focuses on building exactly what was asked for, and avoids excessive labeling or text, even more so than Astra.
1
1
9
3,655
MineBench retweeted
FWIW, here are some MineBench builds Astra generated on release day and today with the same prompt One-off before/afters don't tell you much about stochastic models; if there even was a change, it could just as likely be from changes in the harness (top: release, bottom: today)
Astra (launch day) vs Astra (Today). Same prompt. Today's result looks off. The photo realism is gone. It clearly looks nerfed. GPT-5.6 Sol is unusable for me right now too.
1
1
4
215
GPT-6 Astra Pro is the first model we've tested that seems to be able to handle basically any prompt we've thrown at it. So we tried something a little ridiculous: we increased MineBench's gridSize from 256³ to 8,192³ and gave Astra this prompt: “A full rendition of New York City, fully accurate, dense, and to scale” Astra wrote a ~61 KB program that produced 100.1 million blocks. Expanded into JSON, the build weighed 1.65 GB. That's a 32x increase on every axis, or 32,768x more possible voxel positions. At ~1 meter per block, the canvas spans 8.2 km edge-to-edge. Here's a preview of Astra's build. Note that, at this scale, our renderer struggles to show the entire build.
More GPT-6 Astra Pro generations from community prompts in the gallery! Astra is the first model on MineBench to accurately represent all continents on a globe. View more, or add your own prompts and generations: minebench.ai/gallery
17
29
5
520
54,902
This is personally one of our favorite benchmarks! Inspired by MazeBench, we had GPT-6 Astra build us a maze on MineBench, then tried to solve it ourselves :) We couldn't make it through, but maybe you can: minebench.ai/gallery/gal_i0T…
MazeBench vs GPT-6 Astra Astra spent 60+ hours in this 3D open world spatial reasoning eval. Final score: 14%
7
3
1
41
2,671
And yes, it does have a solution! Here are some renders GPT-6 Astra made of the solution in Codex:
5
301
GPT-6 Astra Pro is now established at an elo rating of 2,315, with a 220-point lead over #2 – the highest rating and largest gap we've seen on MineBench to date!
GPT-6 Astra Pro debuts at #1 on the MineBench leaderboards, with a provisional rating of 2,225! Vote on MineBench.ai to help solidify its rankings!
5
8
151
7,777
More metrics now visible on the leaderboards: minebench.ai/leaderboard iOS app was updated as well :)
The default web-harnesses for most models these days are surprisingly capable (More metrics like this coming natively to MineBench.ai soon :)
2
9
929
Feel free to send us more suggestions!
136
GPT-6 Astra:
Gemini 3.8: a centaur doing a handstand.
1
2
46
7,324
More GPT-6 Astra Pro generations from community prompts in the gallery! Astra is the first model on MineBench to accurately represent all continents on a globe. View more, or add your own prompts and generations: minebench.ai/gallery
6
1
47
44,680