Today, we release our first model, Flint-40B-A2B, a fast local coding model built with our automated research system. huggingface.co/RecursiveSI/f… 1. Strong performance. 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2, and 74.2% on SWE-bench Verified. 2. It is trained from scratch (tokenizer->Pretrain->Midtrain->SFT->RL). 3. Fast in your local laptop. Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark. 4. We build the complete model training pipeline in 4 month!

👉👉 AI Scientific Discovery Demos with our Recursive Flint-40B-A2B 🪨🪨!

A great read. The blog post captures years of progress in mechanistic interpretability and offers an optimistic vision: perhaps we really can open the black box and understand how neural networks work. I share that optimism, but with some reservations. I'm skeptical of explanations that stop at individual neurons, accompanied by compelling visualizations that seem to "explain everything." Humans are remarkably good at convincing themselves. It's easy to cherry-pick evidence that supports a hypothesis, especially when the visualizations look convincing. I'm also not convinced that "Recursive Generalization" is fundamentally different from hierarchical representation learning. As representations become more abstract, they discard details while preserving invariances. Isn't that essentially what hierarchical representations have always been about? IMHO, we need to move beyond a largely descriptive, behavioral approach toward a principled mathematical framework. And we have a natural starting point: gradient descent. Rather than just asking what a network has learned, we should understand why learning dynamics produce these structures in the first place. Of course, this is extraordinarily difficult. Look at OpenAI's 719 mathematical proofs. We see plausible proofs tackling problems at the Fields Medal level, yet no comparable breakthrough that cracks deep learning theory. That contrast is striking. Part of the difficulty is that there are countless plausible theories of how deep networks work. Proving even one can be mathematically formidable. And a proof alone isn't enough. Each theory needs empirical validation, often requiring substantial GPU compute. It's a daunting combination of mathematical and experimental challenges. Still, I believe this is the way forward. Interpretability gives us observations and hypotheses. But ultimately, we need a theory that explains why these phenomena emerge from the learning process itself. That's how we turn compelling stories about neural networks into actual understanding. And hopefully recursive self-improvement can help with that 😀

Thanks for the interview @ziv_ravid! It is a fun discussion 😀

Will be in CoLM conference this Friday afternoon. Won't be able to give a talk (too busy!) but will still join the panel in wdlctc.github.io/efficient-r…. See you all~

We got exact the same comments when we published Coconut😀. However, even with explicit CoTs, the model can still learn to hide the true thinking process. The ultimate solution is always to understand how the model works.

😅 so our StreamingLLM (the original attention sink paper) strikes back again? 😅 -- Caveat: it is in post-training setting.

Interesting findings by our automatic system😆 It finds that some FlashInfer kernels have numerical issues, provides a PR and accepted by human experts🤓

赠Zeyuan Allen-Zhu🫡 世言大模惟精调，君执初心问真章。 动力学中观涨落，表达性里辨行藏。 万里独征云和月，所慕岂在功与赏。 他年若问登临处，格物致理照新航。 突き破れ、扉の向こうへ！ （source: 鋼の錬金術師 ED，《扉の向こうへ》）