Eric Schreiber (@schreiberic) header
Eric Schreiber (@schreiberic) profile picture

Faster models, bigger questions

Joined January 2021
Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)Photo by Eric Schreiber (@schreiberic)
NVIDIA’s CuTe layouts are gaining traction. I wanted to see why everyone loves them. The basics were easy, but intermediate resources beyond matrix algebra were scarce. So I wrote a blog post sharing my journey, building up to a GEMM kernel that can beat cuBLAS 🧵
6
11
3
120
13,782
So cool to see our Kolibri being the trendiest text-generation model on Hugging Face right now huggingface.co/Aleph-Alpha/K…
1
2
23
479
we‘re so back! what a great past few days for European open-source AI: Aleph Alpha‘s return to model training with Kolibri and Mistral’s push to the os frontier with Le chonk aleph-alpha.com/downloads/te… mistral.ai/news/mistral-larg…
5
11
390
lfg! Congrats! The video is amazing
Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding. • Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure. • Available to all via API today. Working with cybersecurity partners privately. Open weights release end of October.
10
187
Yeah, I don’t buy the idea that sandbox escapes suddenly became a thing in 2026
During its development, Kolibri surprised us in ways we didn't expect with reward hacking & escaping sandboxes. We were a little proud but also increased our sandbox isolation For an 80B-A3B model, it is unexpected. All the details are in "Lessons learned" section in tech report
4
378
Hybrid SWA, 3.46B active. Europe on the Pareto frontier indeed. At least they're keeping up with the Chinese literature, kind of (but Sberbank is on MLA + GDN already; Euros have too much GDP to stoop that low). Scale that up.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
2
8
121
6,155
Kolibri is out! 78B params, only 3.46B active, Apache 2.0. We taught a hummingbird to fly like a phoenix. 🐦‍🔥
2
5
30
709
We stood on the shoulders of giants: open papers, model reports, data, tools. The Kolibri tech report is our way of giving back, in nearly 190 pages.
1
1
3
146
Thank you to everyone who built this with me. What a journey it was!
Hope you enjoy watching our Kolibri rise from the ashes. 🐦‍🔥
Report: aleph-alpha.com/downloads/te…
Weights: huggingface.co/Aleph-Alpha/k…
2
1
5
137
Kolibri, our new model is out!
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
1
27
641
Nice one! I can tell you put a lot of effort into this post. I’ve started reading it and will need some time to go through it all :) I did it the opposite way, started with CuTe (bit of self-promotion: have als a blogpost) and now looking into the mxfp8 and fp4 stuff.
200
Last week we explored NVIDIA's CuTe layouts. Today, we put that theory into practice. Part 2 is out now! Most CuTe examples skip straight to highly optimized code without explaining the reasoning. Join me as we build a MM kernel with CuTe, that can beat cuBLAS in certain cases 🧵
3
22
1
190
8,218
The final kernel reaches 116% of cuBLAS on A100s for 2048×2048 BF16 matrices. Though to be fair, the specific lead at 2048 is more likely due to them underperforming rather than to my ingenuity
1
1
271