
PhD-ing @CMU_Robotics | interning @MetaAI, ex-intern @Apple | Vision, Robotics
Pittsburgh, PA
Joined February 2018
- Tweets85
- Following375
- Followers342
- Likes176
Khiem Vuong retweeted
With everyone busily using LLMs + Blender/Three.js to make 3D models, I thought I'd try a different geometric representation.
This dragon isn't a mesh, NeRF, 3DGS, or video model: it's 27kb of GLSL code (generated via LLM), defining a closed-form implicit surface. ๐งต [1/n]
Khiem Vuong retweeted
Current image-to-3D-object methods are either not pixel-aligned (resulting in wrong colours, missing details, and inability to match the input), or bad at generalizing beyond what it is seen.
With @StefanoEsp we fix this, by using rendering-guidance during denoising to pixel-align an otherwise not-aligned image-to-3D-object model, resulting in state-of-the-art results for both seen and unseen parts!
We also extend this to work with multiple input images and video too!
One photo in, a complete 3D object out, faithful in shape, color & pose.
GenIA grounds a frozen 3D foundation model (SAM3D) in the input images at test time. Works from single images, multi-view, even monocular video ๐งต
๐ facebookresearch.github.io/Gโฆ
Khiem Vuong retweeted
Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real.
Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts.
CoRLโ26 ๐ Website for details ๐ prism-real2sim2real.github.iโฆ
Work done at Amazon FAR w/ @zhenkirito123 @pabbeel @rocky_duan @JitendraMalikCV @carlo_sferrazza @ckarenliu @GuanyaShi @akanazawa
Khiem Vuong retweeted
Introducing PointZeroโa 3D world model pre-trained without robots.
Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it?
PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics!
Our model achieves SOTA results on:
๐ฅ Zero-shot 3D dynamics
๐ฅ Action-conditioned 3D dynamics (post-training)
๐ฅ Imitation learning (post-training)
Details and links ๐ 1/6
Khiem Vuong retweeted
World-action models typically imagine the future in RGB โ but are pixels really the right representation for robotics?
Our bet is no: RGB spends capacity on fine-grained details and variation that are often irrelevant for robot policies.
DINO features, point tracks, and depth capture more useful features like semantics, motion, and geometry.
But no single modality captures everything โ how can we effectively combine them?
We introduce โจModARโจ, which predicts the future one modality at a time, with each prediction informing the next. We train from scratch and find that this formulation performs best.
Our 30.1M scratch-trained model even outperforms a 6B video-model-initialized model finetuned on the same data!
๐งตย [1/8]
Khiem Vuong retweeted
[1/N] Introducing Point4D: dense 4D reconstruction across hundreds of frames from a monocular video. Unlike existing feed-forward 4D methods that operate over short windows, Point4D keeps tracking in 3D, even across occlusions and out-of-frame motion.
Khiem Vuong retweeted
My group at Johns Hopkins now has a name and a website
Introducing the Brains, Bots, and Behavior Lab
b3.cs.jhu.edu
We're always looking for creative researchers at all levels to join us!
Khiem Vuong retweeted
Introducing MotionForesight:
A very simple approach re-purposing video models for forecasting future 3D scene flow given a short observed video context
Trained with just 40k RGB human videos, it generalizes well to completely new home/office scenes!
1/n
Khiem Vuong retweeted
To be honest, training on handmade 4D asset datasets is a dead-end. Almost all 4D asset data is synthetic and diverse real data barely exists, so models trained on it struggle to reconstruct objects that deform, get occluded, and move freely about the scene.
Our new work, Lift4D, instead lifts 2D & 3D priors into 4D, reconstructing complete dynamic objects from a single in-the-wild video ๐งต (1/n)
๐Webpage + Demos: lift4d.github.io
Khiem Vuong retweeted
[1/8] Video models generate stunning motion. But can you tell them how bouncy, slippery or soft something should be?
PhyCo (CVPR2026): the same scene under different friction, restitution, deformation, or force โ specified as input, not left to chance. ๐งต
phyco-video.github.io
Khiem Vuong retweeted
Eventful @CVPR 2026 coming up! Presenting some of our latest research on scaling 3D, 4D & World Models ๐จ
My talk at the Image Matching 2026 Workshop June 4th Room 504 1:45 pm LT -
Scaling Representation Learning for Correspondence to Spatial Intelligence! Join for๐ถ๏ธ takes
@PeterHedman3 @RamananDeva talks at the ScanNet++ Workshop on View Synthesis & 3D Worlds - June 3rd R 710 3:40 pm LT
Peter Kontschieder presenting World Modeling research (including stuff from @ethanjohnweber & team) - June 4th R 607 8 am LT, June 4th R 203 2:30 pm LT
@JayKarhade @CMU_Robotics presenting Any4D - June 6th Poster Session 3 ExHall F 11:45 am LT, 4D Vision & 4D World Models Workshop Orals: June 4th R 506 4:30 pm LT, June 4th R 203 5 pm LT
Lastly @OmarAlama @AviBh11 presenting our @AirLabCMU semantic scene understanding research - Findings (June 7th 7:30 am LT ExHall A) & OpenSUN3D Workshop (June 3rd afternoon)
Sadly my first in person CV conference will have to wait ๐ฅฒbut.. do attend for a sneak peek on what we are cooking! ๐๐งต๐
Khiem Vuong retweeted
Presenting two posters at #CVPR this week on vision with light and heat ๐๏ธ๐ท๐ก๐ฅ
Thermal for Image Intrinsics
๐ vt-intrinsic.github.io
๐ Sun 5:30 โ 7:30 PM, ExHall A 518
๐ Wed 11:15 - 12:50 PM, Mile High 4CD
Revealing Heat Flows
๐ Wed 11:15 - 12:50 PM, Mile High 4CD
Khiem Vuong retweeted
Before AI can generate professional videos, it needs to see like a professional.
We spent a year with 100+ content creators teaching AI to describe video like a filmmaker would.
Introducing CHAI: Critique-based Human-AI Oversight for Building a Precise Video Language [CVPR'26 Highlight, Top 3%].
Try prompting a video generator for a dolly zoom, dutch angle, point of view, or camera roll. Most fall back to the same bland defaults: a push-in, a level shot, a third-person view. Why? These techniques require a language of cinema that current models rarely speak.
We built that language:
1๏ธโฃ Precise specification: 5-aspect structured captions co-designed with professional cinematographers covering subject, scene, motion, spatial, and camera dynamics
2๏ธโฃ Scalable oversight: LLMs draft captions, humans critique what's wrong and how to fix it
3๏ธโฃ Post-training recipes: Qwen3-VL-8B surpasses Gemini-3.1 and GPT-5
4๏ธโฃ Video generation: fine-tuned Wan follows 400-word cinematic prompts with precise control
Here's how each works ๐งต
Work led by CMU and Harvard with @chancharikm, @du_yilun, and @RamananDeva.
๐ Paper: huggingface.co/papers/2604.2โฆ
๐ Site: linzhiqiu.github.io/papers/cโฆ
Khiem Vuong retweeted
๐ Excited to announce Vision Banana ๐ and our new paper: โImage Generators are Generalist Vision Learnersโ. We turn Nano Banana Pro into a state-of-the-art visual generation and understanding model.
๐ผ๏ธ Check out our gallery at vision-banana.github.io
๐งต (1/N) continue โฌ๏ธ
Khiem Vuong retweeted
I made a Claude Code skill that generates conference posters ๐ ๏ธ
Instead of a static PDF, it outputs a single HTML file โ drag to resize columns, swap sections, adjust fonts, then give your layout back to Claude. ๐
๐ Skill ๐ github.com/ethanweber/posterโฆ
Khiem Vuong retweeted
๐ฆ LuxRemix๐ฆis out! ๐
Weโre happy to release our project that uses generative modeling to interactively relight indoor scenes (as images or as splats)!
It was amazing seeing our incredible intern @RfLiang lead this effort. ๐
luxremix.github.io/ ๐
Weโre excited to share LuxRemix: interactive light editing for indoor scenes! ๐ ๐ก
Capture a room once, then turn individual lights on/off, change colors, and adjust intensity โ all in real-time 3D from any viewpoint.
๐ก luxremix.github.io/
๐ arxiv.org/abs/2601.15283
Khiem Vuong retweeted
Excited to share recent work with @kaihuac5 and @RamananDeva where we learn to do novel view synthesis for dynamic scenes in a self-supervised manner, only from 2D videos!
webpage: cog-nvs.github.io
arxiv: arxiv.org/abs/2507.12646
code (soon): github.com/Kaihua-Chen/cog-nโฆ
Khiem Vuong retweeted
๐จ The 2nd iteration of our @CVPR Foundational Few-Shot Object Detection Challenge is LIVE!
Can your model think like an annotator? ๐ง ๐ฅ
Align Vision-Language Models (VLMs) with a few multi-modal examples & win ๐ฐ cash prizes!
๐ Challenge: eval.ai/web/challenges/challโฆ
Khiem Vuong retweeted
๐๐๐ง ๐ฐ๐ ๐ข๐ง๐๐๐ซ ๐๐ง ๐จ๐๐ฃ๐๐๐ญโ๐ฌ ๐ฌ๐ก๐๐ฉ๐ ๐๐ฒ ๐จ๐๐ฌ๐๐ซ๐ฏ๐ข๐ง๐ ๐ข๐ญ๐ฌ ๐ก๐๐๐ญ ๐๐ฅ๐จ๐ฐ? Check out our recent ECCV paper "๐๐ก๐๐ฉ๐ ๐๐ซ๐จ๐ฆ ๐๐๐๐ญ ๐๐จ๐ง๐๐ฎ๐๐ญ๐ข๐จ๐ง" to discover how heat reveals form: t.ly/8bGEF
Khiem Vuong retweeted
Replying to @eccvconf






















