PhD-ing @CMU_Robotics | interning @MetaAI, ex-intern @Apple | Vision, Robotics

Pittsburgh, PA
Joined February 2018
Khiem Vuong retweeted
With everyone busily using LLMs + Blender/Three.js to make 3D models, I thought I'd try a different geometric representation. This dragon isn't a mesh, NeRF, 3DGS, or video model: it's 27kb of GLSL code (generated via LLM), defining a closed-form implicit surface. ๐Ÿงต [1/n]
139
475
71
6,245
688,049
Current image-to-3D-object methods are either not pixel-aligned (resulting in wrong colours, missing details, and inability to match the input), or bad at generalizing beyond what it is seen. With @StefanoEsp we fix this, by using rendering-guidance during denoising to pixel-align an otherwise not-aligned image-to-3D-object model, resulting in state-of-the-art results for both seen and unseen parts! We also extend this to work with multiple input images and video too!
One photo in, a complete 3D object out, faithful in shape, color & pose. GenIA grounds a frozen 3D foundation model (SAM3D) in the input images at test time. Works from single images, multi-view, even monocular video ๐Ÿงต ๐ŸŒ facebookresearch.github.io/Gโ€ฆ
2
11
1
89
6,716
Khiem Vuong retweeted
Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real. Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts. CoRLโ€™26 ๐Ÿš€ Website for details ๐Ÿ‘‰ prism-real2sim2real.github.iโ€ฆ Work done at Amazon FAR w/ @zhenkirito123 @pabbeel @rocky_duan @JitendraMalikCV @carlo_sferrazza @ckarenliu @GuanyaShi @akanazawa
12
77
15
374
154,543
Introducing PointZeroโ€”a 3D world model pre-trained without robots. Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it? PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics! Our model achieves SOTA results on: ๐Ÿ”ฅ Zero-shot 3D dynamics ๐Ÿ”ฅ Action-conditioned 3D dynamics (post-training) ๐Ÿ”ฅ Imitation learning (post-training) Details and links ๐Ÿ‘‡ 1/6
8
46
5
261
29,071
Khiem Vuong retweeted
World-action models typically imagine the future in RGB โ€“ but are pixels really the right representation for robotics? Our bet is no: RGB spends capacity on fine-grained details and variation that are often irrelevant for robot policies. DINO features, point tracks, and depth capture more useful features like semantics, motion, and geometry. But no single modality captures everything โ€” how can we effectively combine them? We introduce โœจModARโœจ, which predicts the future one modality at a time, with each prediction informing the next. We train from scratch and find that this formulation performs best. Our 30.1M scratch-trained model even outperforms a 6B video-model-initialized model finetuned on the same data! ๐Ÿงตย [1/8]
12
61
5
411
38,959
Khiem Vuong retweeted
[1/N] Introducing Point4D: dense 4D reconstruction across hundreds of frames from a monocular video. Unlike existing feed-forward 4D methods that operate over short windows, Point4D keeps tracking in 3D, even across occlusions and out-of-frame motion.
6
47
3
331
35,176
My group at Johns Hopkins now has a name and a website Introducing the Brains, Bots, and Behavior Lab b3.cs.jhu.edu We're always looking for creative researchers at all levels to join us!
15
63
5
566
46,849
Introducing MotionForesight: A very simple approach re-purposing video models for forecasting future 3D scene flow given a short observed video context Trained with just 40k RGB human videos, it generalizes well to completely new home/office scenes! 1/n
6
33
4
119
16,679
To be honest, training on handmade 4D asset datasets is a dead-end. Almost all 4D asset data is synthetic and diverse real data barely exists, so models trained on it struggle to reconstruct objects that deform, get occluded, and move freely about the scene. Our new work, Lift4D, instead lifts 2D & 3D priors into 4D, reconstructing complete dynamic objects from a single in-the-wild video ๐Ÿงต (1/n) ๐Ÿ”—Webpage + Demos: lift4d.github.io
11
67
9
463
53,033
Khiem Vuong retweeted
[1/8] Video models generate stunning motion. But can you tell them how bouncy, slippery or soft something should be? PhyCo (CVPR2026): the same scene under different friction, restitution, deformation, or force โ€” specified as input, not left to chance. ๐Ÿงต phyco-video.github.io
2
12
1
61
8,835
Khiem Vuong retweeted
Eventful @CVPR 2026 coming up! Presenting some of our latest research on scaling 3D, 4D & World Models ๐Ÿšจ My talk at the Image Matching 2026 Workshop June 4th Room 504 1:45 pm LT - Scaling Representation Learning for Correspondence to Spatial Intelligence! Join for๐ŸŒถ๏ธ takes @PeterHedman3 @RamananDeva talks at the ScanNet++ Workshop on View Synthesis & 3D Worlds - June 3rd R 710 3:40 pm LT Peter Kontschieder presenting World Modeling research (including stuff from @ethanjohnweber & team) - June 4th R 607 8 am LT, June 4th R 203 2:30 pm LT @JayKarhade @CMU_Robotics presenting Any4D - June 6th Poster Session 3 ExHall F 11:45 am LT, 4D Vision & 4D World Models Workshop Orals: June 4th R 506 4:30 pm LT, June 4th R 203 5 pm LT Lastly @OmarAlama @AviBh11 presenting our @AirLabCMU semantic scene understanding research - Findings (June 7th 7:30 am LT ExHall A) & OpenSUN3D Workshop (June 3rd afternoon) Sadly my first in person CV conference will have to wait ๐Ÿฅฒbut.. do attend for a sneak peek on what we are cooking! ๐Ÿ‘€๐Ÿงต๐Ÿ‘‡
4
16
1
103
18,784
Presenting two posters at #CVPR this week on vision with light and heat ๐Ÿ‘๏ธ๐Ÿ“ท๐Ÿ’ก๐Ÿ”ฅ Thermal for Image Intrinsics ๐Ÿ”— vt-intrinsic.github.io ๐Ÿ“ Sun 5:30 โ€“ 7:30 PM, ExHall A 518 ๐Ÿ“ Wed 11:15 - 12:50 PM, Mile High 4CD Revealing Heat Flows ๐Ÿ“ Wed 11:15 - 12:50 PM, Mile High 4CD
1
8
1,116
Khiem Vuong retweeted
Before AI can generate professional videos, it needs to see like a professional. We spent a year with 100+ content creators teaching AI to describe video like a filmmaker would. Introducing CHAI: Critique-based Human-AI Oversight for Building a Precise Video Language [CVPR'26 Highlight, Top 3%]. Try prompting a video generator for a dolly zoom, dutch angle, point of view, or camera roll. Most fall back to the same bland defaults: a push-in, a level shot, a third-person view. Why? These techniques require a language of cinema that current models rarely speak. We built that language: 1๏ธโƒฃ Precise specification: 5-aspect structured captions co-designed with professional cinematographers covering subject, scene, motion, spatial, and camera dynamics 2๏ธโƒฃ Scalable oversight: LLMs draft captions, humans critique what's wrong and how to fix it 3๏ธโƒฃ Post-training recipes: Qwen3-VL-8B surpasses Gemini-3.1 and GPT-5 4๏ธโƒฃ Video generation: fine-tuned Wan follows 400-word cinematic prompts with precise control Here's how each works ๐Ÿงต Work led by CMU and Harvard with @chancharikm, @du_yilun, and @RamananDeva. ๐Ÿ“„ Paper: huggingface.co/papers/2604.2โ€ฆ ๐ŸŒ Site: linzhiqiu.github.io/papers/cโ€ฆ
25
61
8
369
35,836
๐Ÿš€ Excited to announce Vision Banana ๐ŸŒ and our new paper: โ€œImage Generators are Generalist Vision Learnersโ€. We turn Nano Banana Pro into a state-of-the-art visual generation and understanding model. ๐Ÿ–ผ๏ธ Check out our gallery at vision-banana.github.io ๐Ÿงต (1/N) continue โฌ‡๏ธ
22
70
16
434
62,301
Khiem Vuong retweeted
I made a Claude Code skill that generates conference posters ๐Ÿ› ๏ธ Instead of a static PDF, it outputs a single HTML file โ€” drag to resize columns, swap sections, adjust fonts, then give your layout back to Claude. ๐Ÿ” ๐Ÿ”— Skill ๐Ÿ‘‰ github.com/ethanweber/posterโ€ฆ
30
326
17
2,432
188,720
Khiem Vuong retweeted
๐Ÿ”ฆ LuxRemix๐Ÿšฆis out! ๐ŸŒž Weโ€™re happy to release our project that uses generative modeling to interactively relight indoor scenes (as images or as splats)! It was amazing seeing our incredible intern @RfLiang lead this effort. ๐Ÿ˜ luxremix.github.io/ ๐Ÿ‘ˆ
Weโ€™re excited to share LuxRemix: interactive light editing for indoor scenes! ๐Ÿ ๐Ÿ’ก Capture a room once, then turn individual lights on/off, change colors, and adjust intensity โ€“ all in real-time 3D from any viewpoint. ๐Ÿ’ก luxremix.github.io/ ๐Ÿ“„ arxiv.org/abs/2601.15283
1
14
1,682
Excited to share recent work with @kaihuac5 and @RamananDeva where we learn to do novel view synthesis for dynamic scenes in a self-supervised manner, only from 2D videos! webpage: cog-nvs.github.io arxiv: arxiv.org/abs/2507.12646 code (soon): github.com/Kaihua-Chen/cog-nโ€ฆ
4
33
7
114
24,362
๐Ÿšจ The 2nd iteration of our @CVPR Foundational Few-Shot Object Detection Challenge is LIVE! Can your model think like an annotator? ๐Ÿง ๐Ÿ’ฅ Align Vision-Language Models (VLMs) with a few multi-modal examples & win ๐Ÿ’ฐ cash prizes! ๐Ÿ”— Challenge: eval.ai/web/challenges/challโ€ฆ
1
16
2
36
5,804
๐‚๐š๐ง ๐ฐ๐ž ๐ข๐ง๐Ÿ๐ž๐ซ ๐š๐ง ๐จ๐›๐ฃ๐ž๐œ๐ญโ€™๐ฌ ๐ฌ๐ก๐š๐ฉ๐ž ๐›๐ฒ ๐จ๐›๐ฌ๐ž๐ซ๐ฏ๐ข๐ง๐  ๐ข๐ญ๐ฌ ๐ก๐ž๐š๐ญ ๐Ÿ๐ฅ๐จ๐ฐ? Check out our recent ECCV paper "๐’๐ก๐š๐ฉ๐ž ๐Ÿ๐ซ๐จ๐ฆ ๐‡๐ž๐š๐ญ ๐‚๐จ๐ง๐๐ฎ๐œ๐ญ๐ข๐จ๐ง" to discover how heat reveals form: t.ly/8bGEF
2
22
2
125
12,092
Replying to @eccvconf
@eccvconf #ECCV2020 decisions are now up on OpenReview.
1
7
1
27