Turning Reality into Action.
San Francisco, CA
Joined August 2025
- Tweets17
- Following95
- Followers12
- Likes7
Relantic retweeted
Surprising results from @OpenAI 's new Terra release. It shouldn't be this good! Also separately, adding Clef into the benchmark reveals a new Pareto frontier: decisions are a very cost effective pathway for physical world understanding! (work led by @PerraultAndrew )
Thank you for confirming this! We are seeing the same thing with RealityBench. Still behind on our acceptability metrics, but very surprised to see Argon / Fable / Astra performance at such a strong price point. relantic.com/realitybench
Frontier VLMs have a reality gap: Of 30 models, they now match a human rater at judging physical work, but read a tape measure right only 36% of the time.
We need a benchmark for reality. Introducing RealityBench: 3D worlds you can bend 🧵
Does a simulated world predict the real one? On real photos of tape measures and damaged cars, model rankings track the simulator's.
The worlds are simulators, not image sets: built to extend to new tasks. Early access is on Hugging Face and relantic.com/realitybench Check out our video below.
Relantic retweeted
The pace of improvement has been shocking. I half-expected that the focus on coding and RSI would mean that progress on vision had slowed. Instead, it seems likely that VLMs will be clearly superhuman in performance and cost in the next year or so.
Relantic retweeted
Request early access to environments and images here: relantic.com/realitybench. Paper: arxiv.org/abs/2610.14397. Joint work with @arnabdotorg, @anmoltakiar and Dominik Winecki at @relanticai.
Relantic retweeted
We created simulated environments in Blender for benchmarking VLMs on real world physical work perception and reasoning tasks. Think construction and maintenance of houses and cars.
