PIs: Leslie Pack Kaelbling, Tomás Lozano-Pérez AI/ML/Robotics Research @MIT_CSAIL Website: https://xitter.cf/t.co/TgsPKmvPCb
Joined November 2020
- Tweets139
- Following264
- Followers1.9K
- Likes536
Learning and Intelligent Systems (LIS) @ MIT retweeted
Excited to be at #ICRA2026 in Vienna this week 🇦🇹 -- I'll be presenting SCOUT as a spotlight talk at the Rigorous Robot Perception workshop on June 01 at VIP Lounge C, and SceneComplete as an Oral talk today (June 02 at TuAT3.5).
Learning and Intelligent Systems (LIS) @ MIT retweeted
𝗧𝗶𝗣𝗧𝗼𝗣 𝗶𝘀 #𝟭 𝗼𝗻 𝗠𝗼𝗹𝗺𝗼𝗦𝗽𝗮𝗰𝗲𝘀! Outperforming VLAs including MolmoAct2 and π₀.₅, and WAMs like DreamZero
It's the only method that uses inference-time search and 𝙯𝙚𝙧𝙤 robot data. We didn't do any benchmark-specific tuning.
Learning and Intelligent Systems (LIS) @ MIT retweeted
We ran TiPToP on MolmoSpaces and achieved SOTA! And we wrote an X article thing to discuss details - excited to hear any questions/comments!
Check out the leaderboard here: molmospaces.allen.ai/leaderb….
TiPToP is also fully-open and should take <1hr of setup: try it yourself here: tiptop-robot.readthedocs.io/…
Learning and Intelligent Systems (LIS) @ MIT retweeted
We've released the TiPToP code open source! 🚀
We put a lot of effort into making this easy to run, install and build on. You can get it running on your DROID setup in under an hour. No training data needed. No demonstrations. Just type in what you want your robot to do 🤖💬
You can also adapt it to new embodiments with minimal effort. Give it a try and let us know what you think!
💻 Code: github.com/tiptop-robot/tipt…
📖 Docs: tiptop-robot.readthedocs.io/
Learning and Intelligent Systems (LIS) @ MIT retweeted
Imagine you told a robot to "find your car keys" in your apartment and it looked around, opened a drawer, and retrieved them for you.
As a step towards that, I adapted TiPToP to run on the RBY1 humanoid in our lab! Here's an example instruction it follows: "Put the green block on the blue plate and the yellow block on the magazine."
TiPToP helps plan over the right arm + single torso joint, but it's easy to unlock more joints -- even the base wheels -- for more expressive, real-world tasks.
Humans find objects without thinking twice. One day, robots will too! 🤖
Learning and Intelligent Systems (LIS) @ MIT retweeted
Everyone is scaling VLAs with more robot data.
TiPToP shows another path.
No robot training, no policy learning.
Just RGB + language → 3D scene → GPU TAMP planner → trajectory.
Foundation models + planning alone can run real manipulation tasks.
Learning and Intelligent Systems (LIS) @ MIT retweeted
It was a pleasure to be back at @MIT to present at the #Robotics Seminar! Great to see all the exciting work happening there. Thanks so much @GioeleZardini for hosting me!
Learning and Intelligent Systems (LIS) @ MIT retweeted
Data Flywheel -> Data Avalanche: Thx to Leslie Kaebling and @Pulkitology Agrawal for suggesting "avalanche" as a better metaphor than "flywheel" for combining model-free + model-based methods to bootstrap a specific robot task to bootstrap & amplify on-policy data collection.
Learning and Intelligent Systems (LIS) @ MIT retweeted
#ICRA2025 🤖 I spent 3 years of PhD making efficient long-horizon manipulation planning algorithms. VLMs ultimately provide the essential common-sense and horizon-reduction benefits.
❗VLMs can generate plausible robot task plans, but actions may not be feasible for robots due to reachability and obstacles. 🤝 So we use a TAMP planner to take in the next VLM-generated subgoal and plan for actions and trajectories.
📈 VLM-TAMP successfully solves cooking problems that require 30-50 actions and interact with 20+ objects. Excited to present it on Thursday!
zt-yang.github.io/vlm-tamp-r…
Learning and Intelligent Systems (LIS) @ MIT retweeted
Curious to hear about creating generalist robots from leaders in the field? Don’t miss our panel “Representations for Generalist Robots” (4-5pm) @corl_conf LEAP workshop! Feat. @chelseabfinn @animesh_garg Vincent Vanhoucke @Marc__Toussaint @sidsrivast and Leslie Kaelbling!
Learning and Intelligent Systems (LIS) @ MIT retweeted
Incredible insights from Prof. Tomás Lozano-Pérez of MIT during his keynote on the evolution of robotics! He took us on a journey through the shifting landscape of robotics over the years, discussing “inverted pendulum” theory of Robotics.
#CoRL2024 #RobotLearning #AI #Robotics
Learning and Intelligent Systems (LIS) @ MIT retweeted
How can we get VLMs to help robots solve complex long-horizon tasks? Introducing VisualPredicator: an agent that leverages VLMs to learn predicates and operators for classical planners. Our system can stack blocks, balance weights on a balance beam, and even pour coffee🦾!
[1/9]
Learning and Intelligent Systems (LIS) @ MIT retweeted
Can we get robots to improve at long-horizon tasks without supervision?
Our latest work tackles this problem by planning to practice!
Here's a teaser showing initial task -> autonomous practice -> eval (+ interference by a gremlin👿)
Learning and Intelligent Systems (LIS) @ MIT retweeted
Happy to have our work presented at RSS 2024 in Session 2 Planning!
We build a planning pipeline on Spot for long-horizon mobile manipulation directly in physical world. The key is to decide interdependent cont. parameters of a sequence of parameterized skills on a real robot.
Learning and Intelligent Systems (LIS) @ MIT retweeted
Robot learning has lots of heterogeneity from different sensor modalities (depth/tactile/rgb) and data modalities (simulation/real/human videos).
We illustrate how we can learn and combine information from all these sources by composing policies together!
How to deal with the vastly heterogeneous datasets and tasks in robotics? Robotic data are generated from different domains such as simulation, real robots, and human videos. We study policy composition (PoCo) using diffusion models.
project website: liruiw.github.io/policycomp
Learning and Intelligent Systems (LIS) @ MIT retweeted
Come chat with us on Tuesday morning at #203 at @NeurIPSConf !
Excited to talk about Logic-Enhanced Foundation Models (LEFT) @NeurIPSConf next week! Come chat with us on Tuesday morning at #203.
Try out our Colab notebook to train your own LEFT and learn concepts on a new dataset in ~100 lines of code.
🔗Colab: colab.research.google.com/dr…
Learning and Intelligent Systems (LIS) @ MIT retweeted
There was a lot of good and interesting debate on "is scaling all we need to solve robotics?" at #CoRL23. I spent some time writing up a blog post about all the points I heard on both sides: nishanthjkumar.com/Will-Scal…
Learning and Intelligent Systems (LIS) @ MIT retweeted
[Call for Papers] We are pleased to announce the 20th edition of the “Robotics: Science and Systems” (RSS) conference to be held at the Technical University of Delft in Delft, Netherlands TUDelft, from 𝗝𝘂𝗹𝘆 𝟭𝟱 - 𝟭𝟵, 𝟮𝟬𝟮𝟰.
Learning and Intelligent Systems (LIS) @ MIT retweeted
How can humans help robots improve? Introducing Human-In-The-Loop Task and Motion Planning (HITL-TAMP), a perpetually-evolving TAMP system that learns visuomotor skills from human demos for contact-rich, long-horizon tasks. #CoRL2023
Website: hitltamp.github.io
1/
Learning and Intelligent Systems (LIS) @ MIT retweeted
Introducing UniSim, a generative model trained to be an interactive simulator of the world!
universal-simulator.github.i…
We can train robot policies only on generated interactions, and deploy them on the real robot (with no need for any expensive / unsafe real world interaction)!
Introducing Universal Simulator (UniSim), an interactive simulator of the real world.
Interactive website: universal-simulator.github.i…
Paper: arxiv.org/abs/2310.06114