Rishabh Tiwari (@rish2k1) header
Rishabh Tiwari (@rish2k1) profile picture

CS PhD @UCBerkeley | Ex-Deepmind, FAIR | Research area: Efficient LLM reasoning, scaling RL

Berkeley, CA
Joined May 2019
Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)Photo by Rishabh Tiwari (@rish2k1)
Rishabh Tiwari retweeted
Try it on play.honinglab.com/play/ and to play against one of our best models! We gave open-source decision models two games. It did not go well. 🫠 🎮 Red Alert 2: the base model sold its own base. Then lost. 💥 Tank War: v1 spun in circles. v2 found the fire button… and shot itself. So we let them keep learning: play → train → upgrade the harness → repeat. A few rounds later: model can evolve without human intervention Then we put different models in one arena and let them fight it out. So far, no human has beaten the trained models. 😅 The lesson: a decision model isn't finished when it's released. Real environments keep throwing new situations at it, and a frozen model can't adapt. It needs continuous learning! It's all open source: how we train these models to play, end to end. Next, we're turning it into training infra for any decision model. ⭐ Star it, fork it, or bring your own model to the arena: GitHub: github.com/HectorHHZ/llm-hon… RA2: github.com/HectorHHZ/llm-hon… Tank War: github.com/HectorHHZ/llm-hon… I also appreciate all the suggestions and feedback from: @b_vasilescu , Greg Ganger, @p0nk , @lateinteraction , @matei_zaharia , and @heathercmiller And thanks to all the Contributors to the demo and the github repo!: @Jayzou3773, @brocoliman38951, @ShangyinT , @Xuan04263449 , chenyang yang, tianyu chen , @AlexZhang152005, @Nickzz12138,@JasonWuzh, @Leow_sss730, @ChenMoneyQ , xingyuan ding, @JunchengLi , @LakshyAAAgrawal , @ao_qu18465
14
28
10
68
18,529
Rishabh Tiwari retweeted
How can LLMs learn new skills from offline data without losing what they already know? Introducing Subspace Tuning (SST): a simple approach that improves the learning–forgetting trade-off by controlling where weight updates happen. Paper: arxiv.org/abs/2610.22293 Blog: app.notion.com/p/ac-notes/To… 🧵 1/8
8
32
10
156
16,677
🤖 How well do agents learn from experience? We study agents that amortize experience into reusable artifacts and introduce Agent Plasticity to measure how efficiently they improve on held-out environments. Top performer ≠ most efficient learner.
17
55
16
273
37,225
Rishabh Tiwari retweeted
looking to nerdsnipe some of the best scientists into joining me as operators to make a lasting impact on Gemini's scientific capabilities. know someone? > scientist (life sciences / physical sciences / math) > wants to be an operator at a frontier ai lab > like to be in the weeds of data and evals > agi-pilled > idc if you're experienced or a recent grad (bs/ms/phd) : passion + drive >>> > human written short emails/messages, apply link below
9
16
216
17,830
Rishabh Tiwari retweeted
Happy to share that Interleaved Head Attention (IHA) has been accepted at #NeurIPS2026! 🎉 In standard multi-head attention, heads remain isolated during attention. As a result, combining multiple relationships often has to happen across successive layers. 🧠🔗 IHA lets heads communicate before attention by constructing learned mixtures of their queries, keys, and values, creating richer within-layer interactions while preserving the standard attention operation. ⚙️✨ Across long-context retrieval and mathematical reasoning, IHA consistently improves over standard attention. 🚀 Huge thanks to my co-lead author @dvsaisurya and our amazing coauthors @rach_it_, @rish2k1, @Devvrit_Khatri, @brandfonbrener, and brilliant mentors @pliang279, Inderjit Dhillon, and Manzil Zaheer! 🎥 Quick overview below 📄 Paper: arxiv.org/abs/2602.21371
🧵 [1/14]: Talking-Heads Attention by @NoamShazeer et al. showed something interesting: maybe attention heads shouldn’t be fully isolated. 🧠 That got us thinking: If communication across heads matters, what is the right way for heads to communicate, especially from a one-layer reasoning perspective? 🔗⚙️ That question led us to Interleaved Head Attention (IHA) ✨ 📄 Paper link: arxiv.org/pdf/2602.21371
5
27
2
134
21,953
Rishabh Tiwari retweeted
I got my @Muse to generate a full explainer video with deep research. I think this is an emerging way for thoughtful content consumption. It right now has to work hours to come up with this, and these code generated videos need a place in the mid-training stack! @Musecases
2
1
16
791
Rishabh Tiwari retweeted
Terminal-Bench meetup tonight *with* swag. Register if you haven’t already! Benchmarks, RL environments, new features, and state of the union luma.com/tbench
17
10
3
116
12,394
Rishabh Tiwari retweeted
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs. No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase. Meet Rigel 🧵
23
79
18
636
87,235
Rishabh Tiwari retweeted
nice work from the APEX-Agents team great idea to use GEPA for optimizing judges (agentic verifiers need benchmarks, too!) also... now on the harbor hub!
Today we're introducing APEX-Agents 1.1. As AI models advance, so do their methods to solve APEX-Agents tasks. We’re updating our benchmark with task specifications, tooling, and environments to maintain leaderboard accuracy. Most notably, APEX-Agents 1.1 no longer rewards noncommittal answers. This ensures that our tasks and grading align with how professionals operate. Claude Fable 5.1 remains on top of the leaderboard with 68.6% on Pass@1, though rankings have shifted throughout the rest of the board. Updated rankings for Pass@1: Claude Fable 5.1: 68.6% Gemini 3.7 Flash: 67.8% Claude Opus 5: 65.8% Grok 4.6: 65.3% GPT-6 Astra: 64.7% Read the announcement blog: mercor.com/blog/introducing-…
3
8
1
44
10,168
Rishabh Tiwari retweeted
i can’t stop checking in on this. the marin team is training the largest fully open model ever. 535B params (23B active), 18T tokens. nobody has been this transparent in a hero run before. beyond the weights, the data, logs, and decisions are all open. you can, and should, follow the run live, it’s ~15% through so far! mtracker.oa.dev/hero-run-535…
Marin 535B-A23B is 13% through training. This hero run would not be possible without the generous support of the Jen-Hsun and Lori Huang Foundation, which provided the funding for the compute (Coreweave). Thanks @JensenHuang for supporting open models!
9
57
2
603
62,226
Rishabh Tiwari retweeted
Open-source video generation is now faster than playback without compromising quality. Introducing Video Delta Net (VDN): hybrid attention for live text-to-video with near-lossless quality. VDN accelerates Minimax-H3 by 75 - 90 x, generating 14 seconds of 768p video in 11 seconds on 8× NVIDIA B200 GPUs. Checkpoints + training/inference code + Technical Blog ⬇️ (1/6)
71
171
54
1,232
444,749
The GEPA team went through all feedback we received about prompt/harness optimization runs and used HCI principles to develop deep observability and steerability tools into GEPA/optimize_anything including re-architecting GEPA itself, allowing both you and your agents to know exactly what is going on in the run. The new interface allows you to understand what candidates are being proposed, which datapoints are useful and if any datapoints are misleading or failing repeatedly? With the built-in GEPA agent, you can now chat about the run, inject new candidates on the fly, as well as use @cursor_ai-style brushes to give very targeted feedback to various components like the candidates and evaluators, and see the changes immediately. We are very excited to iterate further on this, and invite you to please sign up for the user study to provide us feedback!
Spent hours digging into GEPA logs finding ways to fix your run? 👀 We have reimagined GEPA to be human-steerable. We built deep observability tools allowing agents to monitor, proactively surface, and fix issues in your run. We provide multiple levers for users to intervene and meaningfully affect the optimization trajectory! Early access signup in 🧵👇🏽
7
22
4
186
14,599
Rishabh Tiwari retweeted
Spent hours digging into GEPA logs finding ways to fix your run? 👀 We have reimagined GEPA to be human-steerable. We built deep observability tools allowing agents to monitor, proactively surface, and fix issues in your run. We provide multiple levers for users to intervene and meaningfully affect the optimization trajectory! Early access signup in 🧵👇🏽
2
13
2
81
19,190
Rishabh Tiwari retweeted
1/ We just made GEPA’s optimization loop much faster, along with improved generalization! Instead of proposing and evaluating one candidate at a time, each GEPA step now proposes a batch of candidates and evaluates them all concurrently. Parallel proposals significantly cut wall-clock time (3–4× faster) while, surprisingly, also generalizing better (up to +11 pts). 🧵
1
10
5
57
25,168
Rishabh Tiwari retweeted
announcing the NeurIPS 2026 workshop on *Transitioning from Pre-training to Post-training*!🍓 come bring your theory, empirics, & methods to enable better foundations, mechanics, & evaluation for LLM training! submit your work by 29th August!🗓️ pretrain2posttrain.github.io… 🧵
5
27
2
165
18,496
Rishabh Tiwari retweeted
Science should not be bottlenecked by access to frontier AI. Mirendil is building systems to bridge that gap - making AI R&D faster, autonomous, and easier to adapt to the hardest scientific problems. A lot more becomes possible when more people can build near the frontier :)
Today, I’m excited to formally announce @mirendil with my amazing co-founders Harsh Mehta, Shayan Salehian, and Tara Rezaei! We’re fortunate to work with @a16z and @kleinerperkins, who led our seed round of $200M, followed by a major investment from NVIDIA, among others. Mirendil exists to accelerate science and technology, and through them, to help solve humanity's most pressing problems. Self-accelerating AI R&D is the most direct path to delivering on AI's broader promise, which is why we believe the most important application of AI is AI itself. Get this loop right, and it compounds. It fundamentally changes the rate of progress itself across all domains. We believe this capability should be democratized. It should be used to power all scientific efforts trying to innovate at the frontier. There are far more important problems—and broader ones—than any single lab can take on, so more groups should be able to pursue them. This pulls concentration of power away from a few labs: businesses and science labs can own their AI and infrastructure, keep their margins, and control their own destiny instead of ceding it all to a single AI lab. We’re a small team with a singular focus. Our founding team consists of 20 researchers and engineers from frontier institutions including Anthropic, xAI, Google DeepMind, and OpenAI, united by a passion for science and a drive to build the technologies that move it faster. If you want to build the system that builds systems, join us! @HarshMeh1a, @shayan_, @tararezaeikh
11
8
1
123
12,974
Rishabh Tiwari retweeted
The teacher distribution matters a ton - just because we expect it to be "better" or "more knowledgeable" doesn't mean it's good to distill its distribution. We show the blindspots of vanilla context distillation, and how it can hurt rather than help.
1
4
17
3,614
Rishabh Tiwari retweeted
Nice summary of learnings on self-distillation in the run-up to our Learning Fast & Slow paper!
3
16
198
43,028