Co-founder and CTO @RewardAI_ | Prev PhD @Stanford
Stanford, CA
Joined January 2021
- Tweets423
- Following871
- Followers4.4K
- Likes3.2K
Pinned Tweet
Can we use wearable devices to collect robot data without actual robots?
Yes! With a pair of gloves🧤!
Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands
Everything open-sourced
Chen Wang retweeted
Super impressive! The co-design of mechanics, sensing, and data are fantastic, and the policy is incredibly dynamic and smooth! Living the dream toward human-level dexterity! Big congrats to @chenwang_j, @zipengfu and the team!
Chen Wang retweeted
That towel flick has personality. Why does the robot make it look so casual?
Chen Wang retweeted
daaamn, just realised this is on 1x speed
smoooth
Chen Wang retweeted
ロボットがバーテンダーの手捬きを再現しているこの映像、人間の作業データだけで学んでいる。しかも映像は等倍。
Reward AIのロボット基盤モデル「OM-1」。一言でいうと、ロボットの学習データをロボットから取るのをやめた。
・センサー付きグローブを着けた人間が普通に作業する、そのデータだけで学習
・テレオペも実機データもゼロ
・卓上アーム / 産業用アーム / ヒューマノイドにゼロショットで転用
・複数ロボットの協調作業まで射程に入っている
ポイントは「ゼロショット」のところ。
従来のロボット基盤モデルは、テレオペで実機を動かして集めたデータが前提だった。だからデータが機体に紐づく。機体を変えれば集め直し。OM-1はデータの入口を人間の手に置いたので、まだ存在しない機体のためのデータを今日から貯められる、とのことだ。
Chen Wang retweeted
Learning from human data could let robot learning scale beyond robot data collection. Cross-embodiment models make this even more interesting - shared skills across different robot bodies. A very promising direction!
Chen Wang retweeted
Amazing!!!
Chen Wang retweeted
This is huge! 🤯
Reliance on real robot data and cross embodiment are big issues of robotics action models! Super excited to see OM1 solving both of them!
The motion is also fast and looks smart!
Great work @zipengfu @chenwang_j!
Chen Wang retweeted
This is really good; very difficult task for a robot
Chen Wang retweeted
Stealth startup @RewardAI_ has unveiled OM-1, an embodied foundation model learned directly from human manipulation data. Zero teleoperation or on-robot data involved.
Spun out of Stanford’s DexCap research, the stack pairs a wearable 7-DoF "Omnibody Hand" with an RL control layer to generalize across tabletop arms, industrial robots, and humanoids.
Chen Wang retweeted
Very impressed by the smoothness and speed of the policy, congrats to friends @chenwang_j @zipengfu @yifengzhu_ut etc at @RewardAI_!
Chen Wang retweeted
Very smooth actions!
Chen Wang retweeted
Really cool stuff! Huge congratulations to the team @zipengfu @chenwang_j @yifengzhu_ut behind this. It’s great to see strong academic research making such meaningful progress in robotics!
Chen Wang retweeted
Really cool to see the smooth motion and it's quite impressive that this is achieved without teleop data. Congrats @chenwang_j @zipengfu and team!
Chen Wang retweeted
1x smooth and fast demos ✅
Hardware aware algorithm ✅
Sensorized human data and co-design ✅
Cross embodiment ✅
Absolutely goated @chenwang_j @zipengfu
Chen Wang retweeted
Lots to watch here beyond the intelligence, speed, and efficiency everyone's noticing; it shows what a truly full-stack robotics team can pull off:
- Human data only. No robot teleop data for post-training.
- Natural human movement, not slowed-down or awkward motions to accommodate the robot.
- Which means the controllers (across different robots) have to be very good to make that data usable.
- Hand pose tracked with electromagnetic sensors, not vision alone.
- Purpose-built end-effector design.
- Tactile + proximity sensing. Proximity is clearly critical for highly dynamic tasks.
- Hardware, data collection, and training co-designed throughout. e.g. the humanoid's head camera sits on the "headphones" (super cute 🩷); my guess is that angle is closer to a human's viewpoint.
Chen Wang retweeted
Very impressive! Congrats to the team!
Chen Wang retweeted
One Model, One Data Interface, Any Body.
@RewardAI_ has released footage of their OM-1 model at work.
Here's what we know from their full release
- Learns new manipulation tasks from under 30 minutes of human demo data, no on-robot training
- Data captured via a wearable "Omnibody Hand" glove with tactile, proximity, camera and electromagnetic sensing
- Claims 9.5mm tracking error at 67cm/s, down from 24.9mm on vision-only tracking
Reward have now entered the foundation model arena. The vertical which has drawn in over $3.9B of funding in the past year alone.
Chen Wang retweeted
Super impressive, congratulations!
Chen Wang retweeted
Addressing the individual components != addressing the systems problem. They really highlight the importance of good model-hardware-controller co-design here, through these very impressive demos.
This will be the hardest challenge for frontier LLM wrappers to overcome!
