building agents and poses @nvidia | Robograd Biorobotics lab @CMU_Robotics | Investing in startups | Taking side of folks who build

San Francisco, CA
Joined June 2009
Check the full suite for full mocap to robotics pretraining . SOMA has anatomically correct joint definitions and has much detailed mesh key points compared to MHR/SMPL. Foundational for all bodypose downstream tasks. More on this soon on its capabilities.
#NVIDIA just released a whole ecosystem for human(oid) motion and robot learning from human data. 🚀🦾 Data, as we all know, is the key to scaling AI models. To accelerate the field of Embodied AI, we have open-sourced a full stack of models and tools to capture, generate, retarget, and simulate human(oid) motion data at scale, along with a massive high-quality dataset and a standard human skeletal representation, SOMA, to make them all seamlessly communicate with each other. The entire suite is available under the Apache 2.0 license. 1️⃣ SOMA: A universal interface to unify all parametric human body models (SOMA-shape, SMPL, MHR, etc.) into a standard skeletal representation, eliminating the need for custom adapters or model-specific retargeting. 🔗 lnkd.in/gsxhiJnn 2️⃣ Kimodo: High-fidelity, controllable text-to-motion generation for both humans and humanoid robots. 🔗 lnkd.in/gCc84XnX 3️⃣ GEM: A global human pose estimation method from in-the-wild videos, natively compatible with SOMA. 🔗 lnkd.in/g_QAvRjn 4️⃣ Bones-SEED: A massive dataset of 150k+ motions in SOMA format, including data already retargeted for the Unitree G1, created with our partners at Bones Studio. 🔗 lnkd.in/gfx-QD-w 🔗 lnkd.in/gyNdTwQx 5️⃣ SOMA Retargeter: A dedicated tool for seamless motion retargeting from the SOMA skeleton to the Unitree G1. 🔗 lnkd.in/gqz9Na-H 6️⃣ ProtoMotions: Our high-performance simulation framework for training digital human(oid)s via RL, now with native SOMA support. 🔗 lnkd.in/gmvMikMU This is just the beginning, and we have much more in the pipeline. Excited to see what the community builds next! #NVIDIA #GTC #GTC2026 #Robotics #EmbodiedAI #PhysicalAI @NVIDIAAI
2
19
3,028
Harsh retweeted
Today, we're sharing a major milestone in our mission to democratize frontier AI development: Pretraining at scale without a pretraining team. We set out to answer an ambitious question: Can we significantly reduce the time, human expertise, and compute needed to pretrain a competitive foundation model from scratch? The result is Mirendil-Base, our first pretrained coding model, built by one researcher in just 3.5 months with less than $10M in compute, including all research, experimentation, and the final training run. Our approach is already showing promising results as we scale to even larger models. This is an important step toward a future where a single researcher, empowered by AI, can accomplish what once required an entire research organization. As these capabilities improve, we believe frontier AI development can become accessible to far more people and teams. More to come soon, including independent evaluations, post-training benchmarks, and API access.
10
21
17
288
61,283
ipdb.set_trace()
one day you wrote console.log("here") for the last time and didn't even know it
1
85
Harsh retweeted
Ask ChatGPT for a new scene, and Swap Anything keeps every movement while changing everything else.
Creative work demands a visual environment. So we partnered with OpenAI to turn ChatGPT into one. Introducing the Pika X ChatGPT plugin extension. All of Pika’s creative tools are now directly in your chat and side panel, in a new interactive creation experience. More on what’s new 👇
9
2
1
36
5,211
Nvidia is the now the most active VC investor in the world, and the largest in Europe. Crazy cc @dealroomco
BREAKING: Jensen has just announced that @nvidia in investing $1bn+ into 5 UK startups: >> Boltz which is building biomolecular models and AI agents that helps scientists predict molecular structures and interactions >> Foresight which helps owners of major infrastructure projects predict completion dates, mitigate risks and accelerate time to revenue. >> Hiverge which is building an AI discovery engine that automatically writes and optimizes algorithms for tasks such as faster AI training and complex logistics planning. >> Prima Mente which focuses on building biological foundation models to decode complex diseases and identify novel therapeutic targets, starting with Alzheimer’s, Parkinson’s and ALS. >> Riverlane which builds real-time error correction hardware and software that work across all major qubit types, helping quantum computers become reliable enough for practical use. He announced the news in London this evening. Apologies for the terrible pic which includes @rowlsmanthorpe’s side profile
12
36
4
264
19,943
NVIDIA just released Kimodo, a controllable human motion diffusion model, on Hugging Face It turns text and kinematic constraints into high-quality 3D human and robot motion, trained on 700 hours of mocap data.
20
72
5
761
33,834
A rare post that takes <10 seconds to read but an hour to understand the depth of it. Tom has written an Econ koan.
One sentence macro model: On the way out the door, every farmer's wife says to every farmer, "the weather's looking bad so don't buy as much at the market this week, but also don't sell our crops for less than the usual price."
11
28
1
1,308
333,767
the year is 20xx and you are asked to renew your neuralink jhana subscription
2
106
Nemotron-Labs models give us a glimpse of what the future might be. And it may be looped! Dory1.0-4B is an impressive 4B model! huggingface.co/nvidia/NVIDIA…
9
22
3
201
17,782
We are excited to announce that we post-trained NVIDIA Nemotron-3.5-Super-VL to be an autonomous ML researcher. We show that our post-trained model goes from 28% to 35% on AutoResearchExam, matching Qwen 3.8 Max, a model with 20× the total parameters. The tasks are long horizon, open ended AI and ML research: building a faster executor for a transformer compute graph, training a RL control policy, creating new uncertainty quantification methods, etc. The model gets to submit once, and it doesn't see its score until the end. We curated SFT data using Self-context distillation. This means that we let the Nemotron model to solve a problem with hints, then we SFT on these trajectories but without the hints. This gave us +11% on AutoResearchExam. As a next step, we added reflection: We ask the Nemotron model to review its own past attempts and writes concrete lessons learned. Then we SFT on these reflections. Surprisingly, training on these lessons improved the performance to +25% compared to the base model. The hardest part was choosing the right data. We selected traces based on task scores, completion checks, and runtime, then refined the sampling across training rounds to address failures in the base and fine-tuned models. We removed hints and filtered out steps that relied on information available only in those hints. The goal was to teach decisions supported by the task and history the model would actually see at inference. Full write-up: bespokelabs.ai/blog/teaching… Thanks to @NVIDIAAI for early access and collaborating on this.
24
55
16
373
101,510
We partnered with OpenAI and basically made an entire editor and all-in-one creative studio inside your ChatGPT. Now your first successful generation with our Pika plugin on GPT gives you 250 bonus credits too! Install now pika.art/mcp/chatgpt/install…
Creative work demands a visual environment. So we partnered with OpenAI to turn ChatGPT into one. Introducing the Pika X ChatGPT plugin extension. All of Pika’s creative tools are now directly in your chat and side panel, in a new interactive creation experience. More on what’s new 👇
23
12
3
188
11,147
We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self. 🧵
58
205
150
1,537
164,643
The is such a wild thread
We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self. 🧵
87
Run Nemotron 3.5 Super VL locally on 64GB Mac 💻 We released 7 quants of @NVIDIAAI's new 120B vision model on Hugging Face from 131.6GB down to 59.3GB For a 64GB Mac we recommend the 59.3GB AD-IQ2_M with a top-1% of 89.8% compared to BF16 GGUFs 👇 huggingface.co/AtomicChat/NV…
Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵
1
8
6
51
9,382
🎆 Nemotron-3.5-Super-VL is here! 🎆 ✅ Same base architecture as Super-3.0: 120B total, 12B active plus Vision encoder ✅ Ahead of Ultra-3.0 on most benchmarks! And now with visual reasoning! ✅ SWEBench-Pro 51.79%, AA index of 26 vs 23 for Ultra-3.0 Post-training pipeline is a greatly simplified and improved version of what we had for Ultra-3.0. We reduced number of teachers, SFT set size while significantly improving data and environments quality and made it multimodal! 1/5
🤖 Made with AI
11
32
5
146
11,087
Harsh retweeted
Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵
🤖 Made with AI
289
553
363
4,447
597,291
its a great day for open models
Very excited to announce our new models: Nemotron 3.5 Super and Lightning! This also feels like a good time to share that a few months ago I joined the amazing Nemotron Pretraining team. We're cooking some very large and very strong models for you! Here is a tech summary for 3.5: * Super is a 120B total / 12B active vision language model (VLM) * Lightning is a 30B total / 3B active text-only LLM * Super scores 51.78% on SWE-bench Pro (better than Nemotron 3 Ultra despite ~4.6× fewer active params!) * arch: MoE Hybrid Mamba-Transformer designed for efficient inference, with a 1M token ctx window, and native speculative decoding (via MTP) * training is done across our GB200 and GB300 (and deployment validated across all imaginable SKUs, including consumer (RTX) and DGX Spark) A few details on how we train these: -- Pretraining * pipeline: initialize with Nemotron 3 base -> custom MTP initialization + 100B-token MTP warm-up -> 1T-token distillation from the next larger N3 teacher * Lightning then goes through a 21B token long-context (LC) stage (@ 1M ctx) * Super goes through: - Indic CPT: 100B tokens at 8K (10B Indic + 90B replay, we built a synthetic corpus with 2.83T toks across 22 Indian langs! Indic benchmarks improve by up to 9.82%) - LC extension: 35B tokens at 1M - Agentic SWE mid-training: 65.4B tokens at 256K (uses whole repos, issue-PR pairs, commit histories, etc.); DeepSWE pass@1 improves from 6.4 -> 10.4 - LC recovery: 9B tokens at 1M (as we observed regression after mid-training) -- Post-training * Super: two-stage SFT -> joint RLVR/RLHF -> specialist teachers -> two-stage MOPD -> MTP boosting * Lightning uses a lighter text-only pipeline: single-stage SFT -> RLVR * Super's SFT transforms the text-only base into a VLM using vision, agentic, reasoning, chat, safety, LC data * A few examples of RLVR environments: - Olympiad-level competitive programming - Finance agents using SEC search, document parsing, evidence retrieval, and tool use * Specialist teachers include Agentic, Reasoning, Video reasoning and tracking, Vision tool-use, Vision-reasoning, and an Alignment/RLHF teacher * Two-stage MOPD consolidates these specialist capabilities into one model * Post-training ends with MTP boosting to better align the drafter and reduce recursive train–inference mismatch; this improves Super's SPEED-Bench acceptance length by 21.8% (to 4.612 tokens) -- Inference The model is highly optimized for inference (with plenty of quantized options to choose from)! A few data points: * On a GB200, Super reaches 4,856 output toks/s/GPU with ≥50 toks/s/user on a 20-turn agentic workload * Super provides 2.34x the throughput of Qwen3.5 122B-A10B and 1.90x the throughput of Qwen3.8 Flash-Next * Native MTP is complemented by standalone parallel drafters: LiLiCorr for Super, and DFlash and DSpark for Lightning Looking forward to seeing what you do with 3.5! Phase shift for Nemotron incoming! :)) (with the usual this will be our weakest model ever, but really really so) paper: research.nvidia.com/labs/nem…
5
228
Very excited to announce our new models: Nemotron 3.5 Super and Lightning! This also feels like a good time to share that a few months ago I joined the amazing Nemotron Pretraining team. We're cooking some very large and very strong models for you! Here is a tech summary for 3.5: * Super is a 120B total / 12B active vision language model (VLM) * Lightning is a 30B total / 3B active text-only LLM * Super scores 51.78% on SWE-bench Pro (better than Nemotron 3 Ultra despite ~4.6× fewer active params!) * arch: MoE Hybrid Mamba-Transformer designed for efficient inference, with a 1M token ctx window, and native speculative decoding (via MTP) * training is done across our GB200 and GB300 (and deployment validated across all imaginable SKUs, including consumer (RTX) and DGX Spark) A few details on how we train these: -- Pretraining * pipeline: initialize with Nemotron 3 base -> custom MTP initialization + 100B-token MTP warm-up -> 1T-token distillation from the next larger N3 teacher * Lightning then goes through a 21B token long-context (LC) stage (@ 1M ctx) * Super goes through: - Indic CPT: 100B tokens at 8K (10B Indic + 90B replay, we built a synthetic corpus with 2.83T toks across 22 Indian langs! Indic benchmarks improve by up to 9.82%) - LC extension: 35B tokens at 1M - Agentic SWE mid-training: 65.4B tokens at 256K (uses whole repos, issue-PR pairs, commit histories, etc.); DeepSWE pass@1 improves from 6.4 -> 10.4 - LC recovery: 9B tokens at 1M (as we observed regression after mid-training) -- Post-training * Super: two-stage SFT -> joint RLVR/RLHF -> specialist teachers -> two-stage MOPD -> MTP boosting * Lightning uses a lighter text-only pipeline: single-stage SFT -> RLVR * Super's SFT transforms the text-only base into a VLM using vision, agentic, reasoning, chat, safety, LC data * A few examples of RLVR environments: - Olympiad-level competitive programming - Finance agents using SEC search, document parsing, evidence retrieval, and tool use * Specialist teachers include Agentic, Reasoning, Video reasoning and tracking, Vision tool-use, Vision-reasoning, and an Alignment/RLHF teacher * Two-stage MOPD consolidates these specialist capabilities into one model * Post-training ends with MTP boosting to better align the drafter and reduce recursive train–inference mismatch; this improves Super's SPEED-Bench acceptance length by 21.8% (to 4.612 tokens) -- Inference The model is highly optimized for inference (with plenty of quantized options to choose from)! A few data points: * On a GB200, Super reaches 4,856 output toks/s/GPU with ≥50 toks/s/user on a 20-turn agentic workload * Super provides 2.34x the throughput of Qwen3.5 122B-A10B and 1.90x the throughput of Qwen3.8 Flash-Next * Native MTP is complemented by standalone parallel drafters: LiLiCorr for Super, and DFlash and DSpark for Lightning Looking forward to seeing what you do with 3.5! Phase shift for Nemotron incoming! :)) (with the usual this will be our weakest model ever, but really really so) paper: research.nvidia.com/labs/nem…
31
18
3
325
13,698
Harsh retweeted
An evening in Paris with Jensen and the founders, researchers, and partners shaping AI across France. 🇫🇷
58
120
24
1,187
92,124
Tonight Jensen met with President @EmmanuelMacron to discuss NVIDIA’s contribution to France’s thriving AI ecosystem through investments spanning startups, AI-native companies, and computing infrastructure. They also covered AI safety and our recently launched Open Agent Safety Platform.
.@JensenHuang wrapped up his Paris visit at the Élysée with President @EmmanuelMacron, discussing NVIDIA’s investments in France’s AI ecosystem, open source, and AI safety. Our road to #NVIDIAGTC Berlin continues.
2
22
138
5,516