Ethan Boneh (@ethanboneh) header
Ethan Boneh (@ethanboneh) profile picture

MTS @RicursiveAI LLM Team, prev @Stanford

Joined August 2022
Photo by Ethan Boneh (@ethanboneh)Photo by Ethan Boneh (@ethanboneh)Photo by Ethan Boneh (@ethanboneh)
I had an incredible time training this model. tldr on some nonobvious things I learned: - Dense models can be surprisingly competitive with much large MoEs in downstream evals! Matching perplexity doesn't necessarily mean matching downstream capability; our dense model often performs better, due to a variety of factors (attention density, less memorization, etc.) - Once you solve your infrastructure and get your synthetic environment pipelines working well, mixed RL is actually extremely efficient at improving your model. Can't wait to continue scaling posttraining! - You can use swarms of small agentic models to turn qualitative questions about your datasets into programmable measurements. This creates a powerful feedback loop for systematically improving data quality and climbing tasks
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
48
16
3
205
10,886
Ethan Boneh retweeted
How do you build a small coding model from scratch? @RicursiveAI 's Dan Zhang breaks down the architecture and decisions behind Ripple-65B Preview with @zainhas: dense over MoE, the ablations that held up (and didn’t), and what happened inside the RL runs. Tuesday, Nov 17 . 11 A.M. PT / 2 P.M. ET
10
2
4
44
8,215
Amazing work!
Introducing Speculative Self-Distillation (SSD, NeurIPS 2026), which pushes the accuracy–efficiency–forgetting frontier of self-distillation. In SSD, student and teacher (the same model with additional information) write the rollouts together. Like a private tutor, the teacher lets the student write and takes over only when the student drifts too far. As a result, SSD trains on rollouts that stay as close to the student's own writing (on-policy) as possible while the teacher keeps them on track. 📉 38–69% fewer training tokens than on-policy, matching its accuracy ⏱️ Reaches on-policy accuracy in ~130 vs ~414 min (Science Q&A) 🧠 Forgets less in a stress test: −1.4 vs −4.5 points for on-policy
1
6
559
Last week we released Ripple-65B Preview. This Friday, come build with it. Ricursive After Dark Hackathon: Fri Nov 13, 3PM–10PM, at our Palo Alto office. Compute, dinner, and hardware prizes. Applications close Thursday, November 12 at noon PT. Accepted participants will be notified by email Thursday night with the address and full details. luma.com/qnsar3a2?utm_source…
11
1
38
5,662
Usually you can drop stronger open-weight models into existing pipelines, tweak/combine pipelines, and/or run task-repair agentic setups. I will say, though, that slightly broken tasks are generally fine for SFT and in many cases can help teach resilience (wrangling with dependencies, looking up documentation, etc), especially given the OpenThoughts-esque results that you don’t need to correctness filter SFT traces. RL is a different story.
Replying to @ethanboneh
hmm if you look at the tasks they open source with the papers they’re always broken tho. by itself oss synth papers have good (sometimes obvious) ideas but very poor QC built into the pipelines you’ll never find the best ideas in the papers
3
4
635
Ethan Boneh retweeted
ripple-65b preview from ricursive is now live on together ai the 65b dense model is built for agentic coding + reasoning, with ricursive reporting 80.5 on terminal-bench 2.1 trained from scratch on 12t tokens, with up to 1m context + apache 2.0 together.ai/models/ripple-65…
6
1
2
19
3,663
Ethan Boneh retweeted
Inkling 2 Small is a great model for post-post-training! Try it out on @tinkerapi!
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…
15
11
111
3,569
Yes!! Made possible by building on the wealth of open source synthetic data literature (details coming soon)
They reached 80.5 on Terminal-Bench 2.1, where RL took just 700 steps with 1.4M rollouts, around 70x fewer than Reflection's Beam. And they bought zero post-training data. It's all in-house synthetic pipelines.
2
59
2,673
Ethan Boneh retweeted
one less surf, one more ablation
I had an incredible time training this model. tldr on some nonobvious things I learned: - Dense models can be surprisingly competitive with much large MoEs in downstream evals! Matching perplexity doesn't necessarily mean matching downstream capability; our dense model often performs better, due to a variety of factors (attention density, less memorization, etc.) - Once you solve your infrastructure and get your synthetic environment pipelines working well, mixed RL is actually extremely efficient at improving your model. Can't wait to continue scaling posttraining! - You can use swarms of small agentic models to turn qualitative questions about your datasets into programmable measurements. This creates a powerful feedback loop for systematically improving data quality and climbing tasks
1
1
12
1,636
Ethan Boneh retweeted
Congratulations to the team! Building a dense 65B model that achieves over 80% in Terminal Bench 2.1 so fast is super impressive! 🍾
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
1
2
13
717
Ethan Boneh retweeted
65B model surpassing models more than 10x the size. another win for open models
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
3
1
9
234
Ethan Boneh retweeted
Achieved with a team of 5!!! 🤯 Talent density unmatched!
1
1
8
962
Ethan Boneh retweeted
⚙️ TESTED LOCALLY: Ripple-65B Preview, a 65B open-weight model from @RicursiveAI, on my 64GB M5 Pro Mac mini It's dense, Apache 2.0 and pretrained from scratch. I ran the community MLX 4-bit build. - Steady 7 tok/s from 512 to 32K tokens - Peaks at 40.5 GB, room to spare on 64GB - 53 for 53 on my local tests, hidden facts out to 64K inc - Runs about as fast as this chip allows for a dense 65B The community had an MLX build up within a day. Good day for owning your intelligence.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
18
8
3
88
10,380
Ethan Boneh retweeted
Ripple-65B from @RicursiveAI is a good model. We tested it ourselves on one RTX PRO 6000. After warmup, in our agent: steady 50-70 tok/s on prose ("write an essay and a poem about regret") and 110-120 on code ("write a simple python script 1000 lines"). #LocalLLM #OpenSourceAI
2
3
24
1,089
Ethan Boneh retweeted
I've tweeted before about how architecture improvements usually improve the constant factor in loss scaling laws, keeping the same exponent. The scaling ladders in the Ripple-65B release from @RicursiveAI are a nice illustration of this at scale! 🧵
5
10
89
5,629
Thanks, Anne! Means a lot
really impressive results! 65B is such an exciting model size. if we can get this level of capability at this scale, a much wider range of hardware becomes more interesting for inference. can't wait to try it out!
2
545
Ethan Boneh retweeted
Excited to see @RicursiveAI launch their first model, Ripple-65B Preview! New U.S. open-weight SOTA on agentic coding with a 65B model. Looking forward to seeing how this advances AI for chip design. Congrats to the team!
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
5
3
18
740
Ethan Boneh retweeted
very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong! beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN! ricursive.com/ripple
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
29
15
3
201
12,700
Ethan Boneh retweeted
Ok, *this is impressive*: Ricursive Intelligence just released Ripple-65B Preview, an open-weight AI model that it says outperforms much larger US models on a key agentic coding benchmark. And even runs locally on my DGX Spark. Ripple scored 80.5% on Terminal-Bench 2.1, narrowly ahead of Reflection’s Beam-501B at 80.1%, according to the companies’ reported results! The size comparison comes with a detail: Ripple uses all 65B parameters for each task, while Beam activates 23B of its 501B parameters. Ricursive says Ripple was trained on 12 trillion tokens with a $32 million compute budget - from scratch. The model supports up to 1 million tokens of context and is available under Apache 2.0.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
88
34
7
515
39,249