Building the world's most diverse real-world, real-workplace, and industry-scale egocentric + robot dataset.
San Francisco, CA
Joined April 2025
- Tweets40
- Following11
- Followers669
- Likes61
Pinned Tweet
Introducing Cortex Harness, a platform for running AI models on real robots with remote human guidance.
Prompt a robot to act. Steer it with language. Let a human take over remotely when needed to keep tasks moving.
Choose from GPT-6 Astra (Ultrafast), Claude, Gemini, open-weight models, and VLAs.
Here’s how it works 🧵
CortexAI retweeted
Anywhere you have an internet connection, you can prompt Dream-1 to control your robot.
Here are rollouts from beta testers from 8 countries, and from our office in SF.
4. Capture every recovery 🎞️
Each intervention records where the model got stuck, how a human helped, and what happened next so models can learn from every recovery.
The goal is a continuous improvement loop: deploy, identify failures, recover with human help, collect recovery examples, train, and redeploy.
We're also inviting researchers, developers, and industry partners to our beta program. Participants will receive $100 in credits.
Read more about Cortex Harness and join our waitlist here 📚
cortexrobot.ai/harness
CortexAI retweeted
This confirms what Twitter demos made pretty clear in the first few days. Great to see an actual eval!
We tested GPT-6 Astra, Gemini 3.8 Flash, and Fable 5.1 on two real-world robot tasks from our MolmoAct2 real-world evaluations:
1. Fitting a 5 mm pipette tip
2. Loading a microwave
Astra scored highest on both. 🥇
Here are some few surprising things that Astra did to come out on top. 🧵
You can see the full results and videos in our blog post here. 📝
cortexrobot.ai/research/llm-…
To train MolmoAct 2 in collaboration with @allen_ai, we created the MolmoAct 2-Bimanual YAM dataset, a 700-hour collection of robot demonstrations involving two machine arms working together.
You can learn more about it here. 📊
allenai.org/blog/molmoact2
CortexAI retweeted
We @cortexairobot tried gpt-6 astra on one of our MolmoAct2 real-world evals: pick up a pipette and fit a 5 mm tip.
It failed on the first attempt, adjusted, tried again, and completed the task.
The neatest part was watching it use its free hand to get a better view.
CortexAI retweeted
We had LinearBot + Wuji hands making coffee for us in the office today ☕️ More mobile navigation + bimanual hands data in the making 🦾
CortexAI retweeted
We're growing the @cortexairobot team across San Francisco, Singapore, and Malaysia.
We're hiring Robot Operator Managers, Robot Operators, and Software Engineers to scale real-world robotics data, deployments, and infrastructure for physical AI.
Check out our open roles: cortexrobot.ai/careers/
Open science robotics foundation models are the fastest way to get robots into the real world.
MolmoAct 2 has now seen 600k+ downloads across its models and datasets.
Cortex AI contributed high-quality, long-horizon real-world data for the bimanual YAM setup and conducted large-scale real-world evaluations across multiple policies.
We'll keep pushing open science to bring robot intelligence out of the lab and into the real world.
Stay tuned.
There are few truly open models in the world, including both weights and data. However, these models are crucial for research and development of new systems — they help us learn which data is important and help develop new capabilities for deploying robots in the real world.
MolmoAct2 provides a foundation for open research into robotics. It is associated with its own open dataset, an open-data action tokenizer, and a reasoning variant which predicts depth tokens. And people have actually been using it across the community, running experiments in their own labs or homes.
@hq_fang and @DJiafei tell us more. Watch Episode 87 of RoboPapers, with @micoolcho and @chris_j_paxton, now!
We provided over 700 hours of robot demo training data towards the MolmoAct2 project, performing tasks like folding a towel, scanning groceries, charging a smartphone, and table bussing.
It still blows our mind to see robot arms successfully interacting with objects they've never seen before! 🤯
One of the coolest things we did this time around: we set up in a public space and asked folks to volunteer their own personal items for MolmoAct2 to interact with. Some of the feedback and results genuinely shocked us — like how it picked up such oddly shaped objects. We're always looking for beta testers, or you can try it yourself since everything is open-source end to end. @allen_ai
🤗 You can find all 700+ hours of the data we contributed to the project fully open sourced here:
huggingface.co/collections/a…