Research Scientist @NVIDIAAI. PhD @UCSanDiego. 4D Vision, Machine Learning, Generative Models.
Joined October 2020
- Tweets147
- Following377
- Followers590
- Likes1.8K
Alex Trevithick retweeted
Excited to introduce reactive optimizers: We let an agent reactively adapt the optimizer code during the training run! The agent acts like a team of scientists constantly choosing the best algorithm, learning rate, momentum, etc, based on its own analysis of step-to-step training dynamics! The results are really exciting. Besides out-performing standard "best-practice" recipes across everything we threw at it, the agent gains a few "superpowers" that no human could have designed into the optimizer... (1/n)
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains?
We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer.
In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans.
📝 xingjianbai.com/reactive-opt…
💻 Try it on your task: github.com/xingjian-bai/opal
🧵
Film once. Move the camera later. 🎥
Our video model reconstructs what you see and generates what you don’t. Feed it a rough 4D reconstruction, get back a dynamic world you can explore in real time!
Led by the amazing @liyuan_zz. Come see how at #NeurIPS2026!
Glad to share that World from Motion has been accepted to #NeurIPS2026! We’re releasing the code, model weights and DyCheck reproduction scripts:
Code: github.com/GradientSpaces/Wo…
Model weights: huggingface.co/liyzzz/World-…
See you in Sydney 🌏🇦🇺🐨!
#neurips2026 #NVIDIA @alextrevith
Alex Trevithick retweeted
GPT-6 Astra recreated the Palace of Fine arts in Blender.
This is favorite building in San Francisco because it was built for the World's Fair in 1914 where the steam locomotive and telephone were showed off. It was a time when technology gave us all a deep sense of optimism for the future.
I feel like some of that sense has been lost since, but I hope models like astra can help restore it, and push us towards a more hopeful future.
Alex Trevithick retweeted
NVIDIA Research is looking for 2027 PhD interns! If you are interested in 4D reconstruction and/or world models, check out research.nvidia.com/labs/amr… and send me an email/DM!
Alex Trevithick retweeted
Mathematics in the age of AI. ~ Terence Tao. teorth.github.io/tao-web/sli… #AI4Math
Alex Trevithick retweeted
How can 4D reconstruction leverage generative priors for unseen dynamics and regions, while respecting the constraints of projective geometry?
Introducing World from Motion: a generative method that reconstructs freely-renderable dynamic 3D Gaussians from monocular video. 🧵🔥
Alex Trevithick retweeted
Introducing MilliVid, our new method for long-context video generation! MilliVid creates videos that are consistent over long time spans, without using retrieval heuristics or 3D maps! (1/n)
davidcharatan.com/millivid/#
Alex Trevithick retweeted
Today, @MichaelElabd, @QuantumArjun, and I are excited to announce Trajectory.
We are a research lab and product company building the platform for Continual Learning.
Our platform unlocks the signal already sitting in product usage, so companies can continuously post-train large-scale agentic models that outperform the frontier. @trajectorylabs
We’ve raised $15M from @Conviction, @BessemerVP, @radicalvcfund, @jeffdean, @drfeifei and more.
We’re partnering with some of the best AI-native companies: @ClayRunHQ @Harvey, @DecagonAI, @mercor, @RogoAI to power their agentic systems, some of which we are already in production with.
We’ve brought together a world class research team from DeepMind, OpenAI, Apple, Meta Superintelligence, Amazon AGI, Scale AI, and an elite product team from Stripe and Figma.
AI will never again start on day one. Every correction, every retry, every edit will make products smarter. This is Continual Learning.
Alex Trevithick retweeted
Collecting demos for training robots is expensive 💸. Each demo is paired with a one-line label that throws away the spatial relations, subgoal structures, etc. The rich pixel information is there — we're just not fully using it.
🚀Introducing DeMiAn: Dense Multi-aspect Annotation as a scaling lever for robot policy learning.
📈 +5 pts on RoboCasa365 tasks, +9 pts on unseen composite tasks
⚡ ~62% less compute at 1M-clip scale (~1.3 × 10²⁰ FLOPs saved)
🤖 Zero new demonstrations collected
Alex Trevithick retweeted
🚀Tired of floaters, flickering, and blur in 3DGS? We introduce a geometry-informed video generator that refines 3DGS renderings in the wild. 🎥✨
We let the video model actually "see" the rendering process using a Gaussian Primitive buffer. #CVPR2026 @CVPR
Project page: research.zhuliyuan.net/proje…
🔥 Highlights:
✅ Geometry-Buffer-conditioned video generator
✅ Refines optimization-based & feed-forward 3DGS
✅ Novel artifact simulation pipeline
✅ Highly efficient bidirectional processing ⚡
Thread 👇
Alex Trevithick retweeted
Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU.
RTFM renders persistent and 3D consistent worlds, both real and imaginary.
Try our demo of RTFM today!
Come check out our workshop at the intersection of generative models and 3D reconstruction at #ICCV2025! 🏝️
📢 SceneComp @ ICCV 2025 🏝️
🌎 Generative Scene Completion for Immersive Worlds
🛠️ Reconstruct what you know AND 🪄 Generate what you don’t!
🙌 Meet our speakers
@angelaqdai, @holynski_, @jampani_varun, @ZGojcic @taiyasaki, Peter Kontschieder
scenecomp.github.io
#ICCV2025
Alex Trevithick retweeted
📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation
Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model!
We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data.
Check out our results in the video (wait for dynamic scenes in the second half!) :
Project Page: research.nvidia.com/labs/tor…
Code and Models: github.com/nv-tlabs/lyra
Paper: arxiv.org/abs/2509.19296
Alex Trevithick retweeted
If you’re at SIGGRAPH 2025 in Vancouver, join us Thu 2 PM for our talk “Generative Neural Materials”! We introduce a universal neural material model for bidirectional texture functions and a complementary generative pipeline. 1/2
Alex Trevithick retweeted
CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
@ChrisWu6080 @RuiqiGao @poolio @alextrevith ChangxiZheng @jon_barron @holynski_
🎥 What if 3D capture could gracefully handle moving scenes and varying illumination?
🎯Come see how video models generate exactly the data you need at our poster, SimVS!
📍CVPR, June 14th (afternoon), Poster #60.
Project page: alextrevithick.com/simvs/
Alex Trevithick retweeted
Poster #60 this afternoon, swing by!
Alex Trevithick retweeted
Interactive looong-context reasoning still has a long way to go. We need progress across all axes: more data, bigger model, and smarter architectures.
∞-THOR is just beginning: generate ∞-len trajectories, run agents online train with feedback and more! Let’s push the limits🚀
Alex Trevithick retweeted
Supervised learning has held 3D Vision back for too long.
Meet RayZer — a self-supervised 3D model trained with zero 3D labels:
❌ No supervision of camera & geometry
✅ Just RGB images
And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy)
🌐 Project: hwjiang1510.github.io/RayZer…
(1/4)