PhD student in robotics manipulation @RiceCompSci. Creator of XLeRobot. Prev Nvidia.
Houston, TX
Joined November 2021
- Tweets525
- Following321
- Followers2.5K
- Likes1.3K
Vector Wang retweeted
Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids.
- learned directly from human manipulation data
- no teleop/robot data
- close to human-level dexterity and efficiency
- multi-robot collab
Vector Wang retweeted
🤖 Introducing RoboTok, the “TikTok for robots.”
Just as TikTok recommends videos to people, RoboTok recommends relevant human demonstrations for robot learning.
Robot learning needs broad and diverse demonstrations, but collecting robot data is expensive. RoboTok is an internet-scale data engine that uses web video as a scalable and continuously growing source of demonstrations for dexterous manipulation learning.
Given one human demonstration video as a query, RoboTok retrieves other web videos with similar underlying manipulation motions.
Rather than matching videos by labels or visual appearance, RoboTok compares how the hands move over time.
💡 The key idea is to represent each video with canonicalized 3D hand trajectories. Each trajectory is expressed in an estimated actor-centered reference frame, so the movement is described relative to the person rather than the camera. This makes similar manipulation behaviors easier to compare across different viewpoints and scenes, even when the actor is partly occluded.
In our experiments, RoboTok retrieved more manipulation-relevant demonstrations than existing robot-data retrieval methods. When those videos were used to guide robot training, the simulated robots completed manipulation tasks more successfully.
I’m sincerely grateful to Howard Qian and Kaiyu Hang for leading this project. I also want to thank Yiting Chen, Kejia Ren, Podshara Chanrungmaneekul, Gaotian Wang, Bowen Wen @bowenwen_me, and Chen Wei @_Chen_Wei_ for their guidance and collaboration.
Project site: rice-robotpi-lab.github.io/R…
Paper: arxiv.org/abs/2609.03199
Code: github.com/Rice-RobotPI-Lab/…
Data and models: huggingface.co/Rice-RobotPI-…
Vector Wang retweeted
We're hiring! The Seattle Robotics Lab (SRL) at NVIDIA is hiring a research scientist to work on robot foundation models. The focus will be on investigating the science behind building VLAs, WAMs, and world models for robotics, and leveraging the findings to propose the next generation of models and training paradigms.
We're seeking a world-class researcher (PhD grad) or senior researcher (3+ years of post-PhD experience) with expertise in training generalist policies. The researcher should aim to question conventional wisdom, push beyond the status quo, and dive into the foundations of general-purpose robot intelligence.
NVIDIA SRL provides a rare environment to conduct open research on fundamental and applied robotics, with the freedom to publish, collaborate broadly, and tackle big scientific questions at scale.
If this sounds like a strong fit, please reach out to us and apply here!
nvidia.eightfold.ai/careers/…
nvidia.eightfold.ai/careers/…
Vector Wang retweeted
"A parcel with snacks has been delivered for Flexion. Retrieve it using the stairs and come up using the elevator. Then unpack it and place the items into the empty drawer on the shelf in the snack area."
One instruction. No human operator. Everything that follows is autonomous.
Today we're introducing Reflect v1.0, our robotics intelligence platform for long-horizon work. From a single natural-language command, the robot understands the task, navigates a multi-floor building, calls elevators, handles doors, uses tools to unpack a box, and puts the items away. The biggest shift in v1.0 is that we use reinforcement learning across every layer, from low-level control to high-level reasoning.
Long-horizon autonomy is unforgiving. The robot must recover on its own when things don't go to plan because in the real world, they never do. Combining reasoning, perception, physical execution and runtime robustness into a single mission-capable system is the foundation required to solve humanoid autonomy.
Our team is just getting started.
#HumanoidRobots #Flexion
Vector Wang retweeted
The most inspiring thing I took from this paper: there's far more to squeeze from simulation than sim-to-real training of task-specific policies. RATs shows a coding agent can self-propose tasks, self-construct scenes in sim, and acquire skills that transfer to real-world deployment.
It's promising to imagine handing coding agents a bunch of simulation clusters on top of ENPIRE to enable Sim-and-Real Co-research, where agents massively learn skills and try ideas in sim while continuously grounding them in the real world. Then robot skill acquisition can really scaling like everything else in the deep learning era.
Children learn from play. Can robots do the same?
We propose 𝐏𝐥𝐚𝐲𝐟𝐮𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐑𝐨𝐛𝐨𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠, a paradigm that gives embodied coding agents a play stage before downstream tasks arrive, and instantiate it with 𝐑𝐀𝐓𝐬 (Robotics Agent Teams), where robots discover reusable skills through curious play.
Co-led with @jiaxin_ge_
Vector Wang retweeted
I’ll be at #ICRA2026 in Vienna presenting ScheduleStream, our work on GPU-accelerated multi-arm task and motion planning: schedulestream.github.io
If you’re looking to meet to chat about NVIDIA, schedule time here: docs.google.com/forms/d/e/1F…
Vector Wang retweeted
🚀 Panthera-HT is here — the cost-efficient robotic arm that doesn't compromise performance. Only $1,799.3.5kg payload | 0.8m/s speed | ≤140μs response Fully open-source (MIT) | LeRobot-native | ROS2 ready
#Robotics #OpenSource #EmbodiedAI #LeRobot #XLeRobot
Vector Wang retweeted
.@NVIDIARobotics is joining the SF Physical AI Hackathon as a sponsor
Every team gets Brev credits to train models on cloud GPUs.
Win the hackathon and walk away with a Jetson Orin Nano Super ! + Brev credits
May 9–10 · @fdotinc Fort Mason, SF
RSVP → luma.com/0xbcm3kj
Vector Wang retweeted
We're building the UI @LeRobotHF should have shipped with
Plug in your SO101 or XLeRobot, scan your ports and cameras, auto-calibrate, and you're teleoperated in under 5 minutes. No terminal.
Vector Wang retweeted
Watch AthenaZero juggle barehanded using on-board sensory feedback only. No motion capture. No funnels. No help adding the third ball. The robot learns to adapt to the uncertainties from contact and the appropriate hand-eye coordination.
Learn more: rai-inst.com/resources/blog/…
Vector Wang retweeted
Happy #NationalRoboticsWeek! Last year, we gave you a sneak peek at AthenaZero, our robotic manipulator built to tackle dynamic tasks like a human arm. Learn more about how this fast, precise robot can switch in an instant from a gentle touch to high force depending on what the task requires: rai-inst.com/resources/blog/…
Vector Wang retweeted
The power of the Claw, in the palm of a robot hand. Agentic robotics is here! Today, we open-source CaP-X: vibe agents, alive in the physical world. They incarnate as robot arms and humanoids with a rich set of perception APIs, actuation APIs, and auto synthesize skill libraries as they go. CaP-X is a strict superset of our old stack, because policies like VLAs are “just” API calls as well. It solves many tasks zero-shot that a learned policy would struggle with.
And we are doing much more than vibing. CaP-X is our most systematic, scientific study on agentic robotics so far:
- We build a comprehensive agentic toolkit: perception (SAM3 segmentation, Molmo pointing, depth, point cloud), control (IK solvers, grasp planner, navigation), and visualization (EEF, mask overlays) that work across different robots.
- CaP-Gym: LLM’s first Physical Exam! 187 manipulation tasks across RoboSuite, LIBERO-PRO, and BEHAVIOR. Tabletop, bimanual, mobile manipulation. Sim and real. Can’t wait to see the gradients flow from CaP-Gym to the next wave of frontier LLM releases.
- CaP-Bench: we benchmark 12 frontier LLMs/VLMs (Gemini, GPT, Opus, Qwen, DeepSeek, Kimi, and more) across 8 evaluation tiers. We systematically vary API abstraction level, agentic harness, and visual grounding methods. Lots of insights in our paper.
- CaP-Agent0: a training-free agentic harness that matches or exceeds human expert code on 4 out of 7 tasks without task-specific tuning.
- CaP-RL: if you get a gym, you get RL ;). A 7B OSS model jumps from 20% to 72% success after only 50 training iterations. The synthesized programs transfer to real robots with minimal sim-to-real gap.
3 years ago, our team created Voyager, one of the earliest agentic AI that plays and learns in Minecraft continuously. Its key ideas — skill libraries, self-reflection loops, and in-context planning — have since influenced many modern agentic designs.
Today, the agent graduates from Minecraft and gets a real job. It’s April Fool’s, but this Claw is getting its hands dirty for real!
Link in thread:
Vector Wang retweeted
ManiDreams is an open-source framework for uncertainty-aware, modular manipulation planning over intuitive physics models.
Great to see it leveraging Newton for physics and Fast-FoundationStereo + SAM2 for 3D perception. 👇
Why can't “world models” just be sim? A real-time real2sim demo in the upcoming “ManiDreams”
Foundation Stereo + SAM2 + Newton
= Generalizable 3D physics prediction in a fast (15FPS) and lightweight (3070) way.
No data or object mesh needed
rice-robotpi-lab.github.io/M…
Vector Wang retweeted
Introducing ForeHOI - One small step toward Physical AI! Accepted to #CVPR2026 🎉
It turns real human experiences — the natural way you touch, grab & use objects every day — into 3D instantly. This will equip robots to perform precise sequential actions in the physical world 🚀
Vector Wang retweeted
We're introducing EnvHub to LeRobot!
Upload simulation environments to the @hugginigface hub, and load them in lerobot, with one line of code !
env = lerobot.make_env("username/my-env", trust_remote_code=True)
Back in 2017, @OpenAI called on the community to build Gym environments.
Today, we're doing the same for robotics.
We're calling researchers, students, hobbyists, labs, and startups to publish complex envs, tasks, and benchmarks on the Hub, so we see how robot policies perform. (in Isaaac, Mujoco, Genesis, etc..)
Fill out and submit the form linked in the comments to join the effort!
Vector Wang retweeted
finally got around to implementing the software for teleoping the detachable gripper on top of @LeRobotHF
going to release open source files for it soon
Vector Wang retweeted
Running a GR00t N1.5 model for matcha pouring. Getting ready for hackathon @seeedstudio
@NVIDIARobotics @VectorWang2
Vector Wang retweeted
Exciting things at LeRobot! 🤖
We’ve just integrated Meta-World: a benchmark for testing multi-task and generalization abilities in robotic manipulation: huggingface.co/docs/lerobot/…
We’ve also cleaned up our environments and standardized on Gymnasium ≥ 1.x.x and MuJoCo ≥ 3.0.0 across the board.
Train your policy with the Meta-World dataset → huggingface.co/datasets/lero…
All in LeRobot! next is making environments as easy to load as datasets or models 🚀
Vector Wang retweeted
🚀 All teams are ready for the upcoming #EmbodiedAIHackathon — a 2-day onsite build for home & cooking #robots! ✨
👉 seeedstudio.com/embodied-ai-…
Project proposals include dual-arm kitchen assistants, boba-making bots, language-guided helpers, and even home doctors — all integrating sensors, #VLA/#GR00T N1.5, real-time inferencing on @NVIDIARobotics Jetson #Thor, and robot arms like @huggingface @LeRobotHF #SOARM101 & #FashionStar StarAI.
Also great joining: mentor teams @LightwheelAI & #XLeRobot @VectorWang2 for advanced tech guidance support.
Let’s build and shine with #PhysicalAI! 🌍🦾
