Postdoc @berkeley_ai. Researcher @IBMResearch. PhD @TelAvivUni. Working on Multimodal Robot Foundation Models and Structured Physical AI.

Berkeley, CA
Joined March 2017
If you want a vision encoder for dexterous manipulation, what should be the most important part to model? 🤔 Current standard models like CLIP, SigLIP, and DINOv2 have an incredible grasp of semantics and spatial details. But they lack the action-centric structure needed for downstream visuomotor control. But collecting annotated robotic trajectories at scale is SUPER expensive and largely unrealistic. So, how do we bridge this gap? We introduce CAIP (Contrastive Action-Image Pre-training) ⬇️ 🔸 Action-centric upstream: we align visual observations with action chunks through a contrastive objective. 🔸 Human video as a proxy: we represent 3D human hand poses analogously to robotic end-effector actions, tapping into a massive source of human demonstrations. 🔸 Massive scale: pre-trained on over 32,000 hours of manipulation video, driving both sample efficiency and robust generalization. 🔸 Hardware proven: achieves a 76% average success rate on a real-world Dexmate Vega bimanual @DexmateAI manipulator with dual 22-DoF Sharpa Wave hands @SharpaRobotics . 🔸 State-of-the-art: significantly outperforms strong baselines like DINOv2, SigLIP, MVP, and Qwen3.5 ViT across complex tasks, even under unexpected lighting changes and visual distractors. 🌐 Project: caip-encoder.github.io/ 📝 Blog: xitter.cf/roeiherzig/status/2063… 📄 Paper: arxiv.org/abs/2606.17256 💻 Model: huggingface.co/yuvansharma/c…
5
22
1
148
27,115
Our new paper shows how coding and action prediction can be unified in a single model! Check this out! Amazing work by @liu_zhuoyang13 and @Dantong_Niu
We're excited to introduce AW-0: Agentic World Action Models! 🤖 (0/8) The first World Action Model that plans in code, predicts the future, and controls the robot—all in one generative model. AW-0 jointly generates: 🧠 Code - explicit skill calls, object arguments, and control flow 🎥 Future video - predictive visual dynamics 🦾 Robot actions - continuous motor control All three are generated together through joint denoising, letting semantic plans directly shape both predicted futures and robot behavior, with: 🤝 Hybrid execution — generated programs can also run directly through perception and control APIs ⚡ Just 40 ms of additional inference latency over the base WAM on an RTX 5090 📚 Pretrained on ~3,000 hours of robot + human demonstrations One model for planning, prediction, and execution. 🌐 Project: agentic-wam.github.io/ 📄 Paper: arxiv.org/abs/2610.30136v1 💻 Code: github.com/ZhuoyangLiu2005/a… 🤗 Model: huggingface.co/miniFranka/ag… 🧵 How it works—and what we found ↓
1
94
Very, very cool! But if you want International academic labs to participate, you’ll need to provide the hardware...
The AI physical Olympics will be held in Tel Aviv 🇮🇱!! Details and registration below! Tel Aviv, May 18–19, 2027. 10,000 people in the arena. Forget scripted, pre-recorded demos where the robot succeeds on take 47. This one will be live! The P.AI Games—organized by the Israeli National AI Directorate | Prime Minister's Office and led by Guy Altagar—is putting embodied AI to the ultimate test: 10 real-world arenas. $1.35M in prize money. Live scoring in front of an arena crowd. Here are the events: → Pick up oily crystal glasses and pack a dishwasher in under 8 minutes without cracking one. → High-speed dynamic baton handoff between two moving robots across a 20-meter track. → Face off in real over-the-board chess against another robot, moving pieces and hitting the clock. → Balance a raw egg and a full tray of water up an incline ramp with zero spills. → Grab a 14-lb bowling ball and take down a full rack of pins. Humanoids, quadrupeds, wheeled bases, custom manipulators—every morphology is welcome. Whether you want to dominate a single niche challenge or take on all ten, this is where the bar gets set. No entry fee. Plus, participating teams receive dedicated financial support to make the trip happen. If you run a robotics startup, lead an OEM team, or build cutting-edge embodied models in academia, this is your arena. Drop a comment or DM me if your team wants to throw their hat in the ring—I'd love to see you on the floor. Details and registration: pai-games.com/
4
610
The winner for today 🏆
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
2
385
Very (very) cool! I wonder how many opportunities this creates for academic labs through an open API.
Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour. Dream API is live for everyone, today. 🧵
5
577
Playing for the win! Nice work 🥳
We taught a robot to play! Introducing Dexterity from Supervised Play (DSP): turning minutes of human play into a world model and dexterous manipulation policy. No task demonstrations or sim2real! play-policy.github.io
8
1,159
That’s not true. Capability should come with efficiency. Have you seen a kid grasping all the objects in the world before he manipulates a new object? Like compression in information theory, once you’re data-efficient, you’ve captured the structure that leads to generalization.
Replying to @YouJiacheng
first comes capability, then efficiency! we've already managed to bring compute requirements down, stay tuned :D
3
2
1
26
3,582
What is going on with Berkeley's weather for this week?
356
Welcome back!!
Announcing Gemini 4 Argon, our new frontier model. Argon is built to sustain deep reasoning across complex, long-horizon workflows and delivers frontier performance in complex workflows across real-world software engineering, enterprise knowledge work like legal and finance, and cybersecurity defense. We’re also expanding the model’s output token limit to an industry-leading 1M tokens. Argon is currently rolling out to a set of trusted cyber defenders in the Fairwind Program, with broader availability as soon as possible.
1
4
829
Amazing opportunity to continue pushing the robotic harness frontier! Don't miss it. There are very few places and labs where you can do it.
If you're at #IROS26, completing your PhD soon, and interested in a Postdoc position to collaborate with me and my students on Agentic Robotics for bimanual manipulation in the AUTOLab @UCBerkeley, please come talk with us at the Bistro near the Westin Hotel Registration Desk between 7-8pm tonight! We're particularly interested in PhDs with skills in model-based methods, control theory, computer vision, and motion planning who are excited to work with frontier LLM and VLM models!
1
891
Maybe just Findings?
Maybe computer vision conferences need something like an "AI methods" track. This would not be a traditional paper -- it would describe how you leveraged a frontier model to solve an outstanding problem. The publication would describe the problem, give the prompts, resulting code, demo, etc. This would allow the field to say "ok, that problem is solved sufficiently well" and move on. This acknowledges something that's been bothering reviewers for a long time. Reviewers feel papers require technical novelty. If one uses a frontier model, there may not be anything technically interesting. But the fact that an open problem is solvable, and how you solve it, is interesting. I would not have a full review process for such things. An abstract and a link to a demo might be enough. People would get a poster or demo space to show what they did. I can imagine that practical sessions where people get insights about how to solve problems today could be interesting. Just a thought.
1
2
488
Congrats to everyone 🎉
We are excited to announce that World Labs is joining @AMD. The research and technical breakthroughs we have achieved since our founding in 2024 have given us a clear vision for AI’s potential to solve problems in the spatial and physical world. Accelerating the future of spatial and physical intelligence requires scaling our efforts, scaling our reach, and getting closer to the hardware.
5
753
💯
Yep. This is a toxic (and crazy) outlook. There are plenty of important problems that basic research will solve outside of labs. Much of this pessimism btw is created by people that have never experienced excellence in pure research and thus don't have a frame of reference.
2
969
Best Sukkah ever!
Na I can’t do this anymore
Readers added context they thought people might want to know
The photo is AI-generated. Replies and quotes identify it as such; no records exist of this exact sponsored sukkah event. x.com/_djpn/status/2… x.com/MattHasTheShoe… forward.com/culture/520812… techcrunch.com/2026/08/04/ben…
1
612
If Codex is down, does #ICLR allow a 1-day grace period? These days, it's worse than if OpenReview is down.
We are aware that codex is down and are working hard to bring back normal service.
4
925
Yep.
there are many many people who studied physics and came to the conclusion that working on machine learning would probably create more scientific progress than studying physics including many of the original chatgpt crew like john schulman, liam fedus, amodei & jared kaplan at ant
409
כל הפיד שלי מלא בישראלים שמתלהבים ממיוז משום מקום ובלי שום קשר. איזה מוזר זה, במיוחד שאף אחד עוד לא התחיל באמת להשתמש בזה פה.
תוך כמה זמן Muse מגיע ל 100 מליון משתמשים?
63%עד רבעון
25%עד שנה
12%יותר משנה
354 votes • Final results
7
1
2,729
I've been playing with @Muse recently, but still haven't figured out the best way to use it. What are the good use cases people are actually using it for? For example, it refuses to control my WhatsApp and send automatic messages, but it sends on Instagram, which is really not useful.
3
695
The kind of dataset whose impact only becomes obvious 1–3 years later. HUGE RESPECT TO THE TEAM BEHIND IT.
Full ABC release 🔤 ! Train, test, and evaluate your models. The full suite of simulation tasks and data has also been released! Excited to see what people build 🔨🤖
1
10
2,366
Every few years the field agrees the review system is broken. Then submissions double. ICLR just passed 50,000 🤯 At some point we have to cap how many papers an author can submit ✋📄...Nobody has more than 5 conference-worthy ideas in a single cycle.
15
6
4
171
30,530
🚨 Call for participation: the Child–Robot Dexterity Challenge at the CoRL 2026 𝗦𝗣𝗜𝗡 𝗪𝗼𝗿𝗸𝘀𝗵𝗼𝗽! Bring your robot, or run on ours. Apply by Oct 5. 🧒🤖 Robots and a child will take on the same everyday tasks, side by side, live on stage. The demonstration offers a simple way to see what today’s robots can already do and where there’s still room to improve. We make it easy to join: - Bimanual YAM setup (MolmoAct 2 style) + a dexterous Sharpa hand setup, provided on stage. - Bring your own hardware (subject to approval). 🌐 Website: spin-workshop.github.io/#con… 🗣️ Speakers & panel from Stanford, Columbia, TU Darmstadt, and AMI Labs. 📍 Nov 12 · Austin, TX #CoRL2026
2
12
1
36
5,924
Possible tasks: grasping, opening bottles, insertion, folding, and pouring. Every task will be child-appropriate. We’ll share the final list and completion guidelines in advance.
1
3
410
Academic and industry teams are welcome. Bring approved hardware, use a provided setup, or contribute hardware, data, or sponsorship. Apply by Oct 5: [email protected], [email protected] @_amirbar , or [email protected]. @carlo_sferrazza
5
349