Benchmark what robot models can actually do in the real world. Free eval hours for academia ↓
Joined October 2026
- Tweets84
- Following51
- Followers1K
- Likes161
Pinned Tweet
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
More open-source tools for robotics builders are available than ever before. Now, we’re adding open evaluation to the mix!
With @huggingface, we’re sharing RoboColosseum’s real-world runs, including failed attempts.
Researchers: apply for a share of 10,000 free eval hours ↓
Open-source is critical for robotics AI. It means a student, a researcher or a startup can build on years of progress and contribute something the whole community can use rather than concentrating everything in a few companies building a black-box robot.
That’s why we’re building @huggingface into the platform for open-source robotics AI! Amazing to see the progress we've made in the six months and it's accelerating!
- Almost 100,000 public robotics datasets on hf.co/datasets, up 48%
- Over 30,000 public robotics models, up 41%
- Microduck and Reachy mini, our open-source robots became the most popular robot of 2026 (probably ending the year with 50,000+ sold and 20,000+ shipped)
- We’re a founding partner of @RoboColosseum, evaluating robots across 140+ real-world scenes. Every run is published publicly on HF, including failures, so everyone can learn from what actually happens.
And @LeRobotHF got downloaded over 1M times and v0.7.0 just landed with new world action models, interactive robot control and remote GPU inference: huggingface.co/blog/lerobot/…!
Huge thanks to everyone making this happen 🤗🦾
If you’re scaling real-world evals for a paper beyond your budget, apply for support at RoboColosseum: app.youform.com/forms/riubyc…
RoboColosseum retweeted
I couldn’t ask for a better team to build @RoboColosseum with:
🤗 @huggingface for advancing open-source robotics
🌏 @eaic_sg for making Singapore a global hub for robotics evaluation in the wild
🤖 @BitRobotNetwork for the expertise to run real-world robot evaluations at scale
🦾 Together, we aim to bring clarity to robotics through evaluation.
Open-source is critical for robotics AI. It means a student, a researcher or a startup can build on years of progress and contribute something the whole community can use rather than concentrating everything in a few companies building a black-box robot.
That’s why we’re building @huggingface into the platform for open-source robotics AI! Amazing to see the progress we've made in the six months and it's accelerating!
- Almost 100,000 public robotics datasets on hf.co/datasets, up 48%
- Over 30,000 public robotics models, up 41%
- Microduck and Reachy mini, our open-source robots became the most popular robot of 2026 (probably ending the year with 50,000+ sold and 20,000+ shipped)
- We’re a founding partner of @RoboColosseum, evaluating robots across 140+ real-world scenes. Every run is published publicly on HF, including failures, so everyone can learn from what actually happens.
And @LeRobotHF got downloaded over 1M times and v0.7.0 just landed with new world action models, interactive robot control and remote GPU inference: huggingface.co/blog/lerobot/…!
Huge thanks to everyone making this happen 🤗🦾
The first RoboColosseum results are in.
We tested five robot models and three frontier AI agents across 124 real-world sites.
Here’s what researcher @YiruHelenWang found about what today’s models can do and where they struggle ↓
4/ A model that leads in one evaluation won’t necessarily lead in another.
Across RoboColosseum’s tests, rankings shifted between robot-and-task suites and between the Open and Fine-tune Tracks.
One leaderboard can’t tell the whole story of what a model can do.
5/ The takeaway: robot performance varies widely across tasks and hardware.
To understand what works, we need to test at scale and look beyond a simple pass or fail. This is just a snapshot of the evaluations that we will keep running at @RoboColosseum.
Explore the results and open data: robocolosseum.ai
Paper: arxiv.org/abs/2611.02497
Apply for FREE eval: app.youform.com/forms/riubyc…
Thanks for the technical dive @YiruHelenWang!
🤖 Introducing RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild!
We ran 5,699 real-world robot comparisons across 124 sites, evaluating 5 VLA policies and 3 frontier AI agents.
What did we learn about the state of robot foundation models and agentic models? 🧵⬇️
#robotics #ai #benchmarking #evaluation #foundationmodels #agents
RoboColosseum is live. Next stop: @corl_conf.
See real-world robot evaluation in action at the @BitRobotNetwork booth (408), Nov 9–11.
Join us for demos and lightning talks from @DJiafei @pepijn2233 @YiruHelenWang,@ruijie_sg and more.
RoboColosseum retweeted
Robot models that wanna be the very best,
like no one ever was
To work IRL is their real quest,
To train them is our cause.
Say hello to eval gym battles with RoboColosseum!
Singapore has been quietly building up to its embodied AI moment and it's wonderful to witness.
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
RoboColosseum retweeted
Most robotics teams don’t have a fleet of robots and real-world test sites at their disposal. RoboColosseum is bringing those resources together for researchers.
The open evaluation platform has a network of 100+ robots and 140+ real-world environments. Teams can apply to have their robot policies tested.
Evaluation runs are published on Hugging Face, failures included, for other researchers to study.
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
RoboColosseum retweeted
Robotics finally has its real-world arena.
RoboColosseum is a real-world benchmark that tests robot policies on hundreds of actual robots across 150+ scenes, not cherry-picked demos. You bring a model, they run it blind on real hardware and publish every rollout (failures included) on Hugging Face.
Two ways models are tested:
- Open track: two unnamed models get the same brand-new instruction in the same scene. A human judge watches both runs and picks the winner, or a tie. Think LMArena, but on real robots. It measures zero-shot performance on novel tasks.
- Fine-tune track: every model is fine-tuned on one fixed task from the same open-source demos, then gets 25 attempts. An apples-to-apples test of how well a model learns a specific skill.
Over 30 policies from OpenAI, Anthropic, NVIDIA, Galaxea, Ai2 and more are tested across Unitree G1, Franka, YAM and SO-101.
Why it matters: across 20,000+ real runs, no model won everywhere. Rankings flipped depending on the task and the robot body. That's the whole point: a single demo or benchmark tells you almost nothing about what a policy can actually do. This is the field finally getting honest, real robots, in the wild, every failure published.
And 10,000 eval hours are free for academic labs.
robocolosseum.ai/
RoboColosseum retweeted
💎 Thanks again to Diamond Sponsor @BitRobotNetwork! 🤖
The robots are connected, and the models are getting tested. They are supporting large-scale real-world benchmarking with @RoboColosseum. More news at #CoRL2026 teapawt.org/RoboColosseum/status/2…
RoboColosseum retweeted
Why do other fields in AI seem to make more meaningful progress than robotics? Much of it comes down to benchmarks.
When we hill-climb on the right benchmarks that reflects real-world progress then we can tell what truly works and what doesn’t.
@RoboColosseum aims to tackle the hardest part of robotics: real-world evaluation at scale in-the-wild!
RoboColosseum retweeted
The Hugging Face for robotics is being built by Hugging Face !
Robotics needs open benchmarks, evals, and leaderboards. Benchmaxxing is a proven way to build better models. Not the only way, but it’s very important.
RoboColosseum (robocolosseum.ai) tests real robots on tasks like stacking bowls, opening drawers, folding towels, loading a washing machine, or carrying a box.
Tasks are performed on Unitree G1s, dual-arm YAMs, Frankas, and SO-101s, in real homes, offices, and public spaces.
Open track uses fresh on-the-spot instructions. Fine-tune track gives every model the same 100 demos, then scores 25 attempts.
Public leaderboards + all data on Hugging Face.
I’ll be posting regular updates on leaders and leaderboard changes.
Thank you to @huggingface @RoboColosseum @BitRobotNetwork @DJiafei @eaic_sg for building this!
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
RoboColosseum retweeted
How well do robot policies generalize beyond the lab?
Excited to share RoboColosseum, our effort to scale real-world evaluation across robots, tasks, and environments. 5,699 pairwise comparisons across 124 sites, and plenty still to learn!
robocolosseum.ai/
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
RoboColosseum retweeted
Robotics has a massive evaluation bottleneck. Most benchmarking efforts measure model progress inside pristine lab environments or simulated worlds, but real physical utility requires operating reliably inside unconstrained home environments.
RoboColosseum just launched to solve this as an open evaluation platform backed by Hugging Face, BitRobot Network, EAIC, and DJiafei. It deploys over 100 hardware units across 140 real world scenes, providing 10,000 free evaluation hours for academic research labs.
RoboColosseum retweeted
Big step…
Robots needs large-scale real-world testing.
RoboColosseum is building a distributed benchmarking network, evaluating robots across four embodiments, including Unitree G1 humanoids, with 10,000+ physical trials.
One interesting finding--model performance can vary significantly across tasks and robot embodiments. Success on one platform doesn’t guarantee success on another.
I also think customer deployments will become an important long-term benchmark for robotics.
How reliably can robots perform over time? and they consistently complete tasks, adapt to real workflows, and deliver measurable value in actual operating environments…
These deployments generate a different kind of evidence, accumulated through months and years of real-world use.
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread ↓
RoboColosseum retweeted
41 days. 124 diverse sites. 11,398 real-world evaluations.
At @RoboColosseum, we compared 5 VLA policies and 3 frontier AI agents in the wild. One clear takeaway: @OpenAI’s GPT-6 Astra is seriously strong. Those demos on X line up with what we’re seeing at scale.
Read our 34 pages technical report for many more interesting insights: arxiv.org/pdf/2611.02497