Lukas Blübaum (@LukasBluebaum) profile picture
Joined September 2025
Lukas Blübaum retweeted
Expert load balancing is key to training Kolibri efficiently at scale. We developed two new methods: Exact Quantile Balancing (EQB) for global balance and Load-Error Injection (LEI) for local balance. @JesseLiJiaze and @PitNeitemeier explain them in a new, visual blog post.
1
11
5
43
5,373
Lukas Blübaum retweeted
313
439
164
8,619
565,039
Lukas Blübaum retweeted
Introducing North 2. Enterprises have been compromising on agentic AI for years: platforms that aren't scalable, models that aren't sovereign, control without enterprise-grade security. That ends today, with 15+ new features in our biggest upgrade yet:
43
55
32
813
97,694
Lukas Blübaum retweeted
My favorite reinforcement learning plot from our Kolibri report: Inference v.s. training log probs, background is the logit gradient norm of importance-sampled Reinforce. Numerically unstable for small inference log-probs, but you can do exact token-level gradient clipping in RL.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
2
5
2
27
1,165
Lukas Blübaum retweeted
New Apache 2.0 model from aleph alpha. It rocks. This is the way. So excited to build together.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
8
11
120
5,027
Lukas Blübaum retweeted
New Apache 2.0 model from @Aleph__Alpha. It's really good. If you like that, you're going to love what we build together.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
16
21
3
242
11,198
Lukas Blübaum retweeted
Proud of what we achieved together. We’ve shipped a good model, but also built an incredible team that’s moving at high velocity. Kolibri climbs faster than Kolibri Origin. Excited for what’s next.
Replying to @Aleph__Alpha
Kolibri went from the start of pre-training to taking flight today in less than two months. We built the foundations from January. From the beginning, we focused on fast iteration and on building a pipeline for the long term. This was enabled by our Model Factory, which orchestrates the Kolibri training and evaluation, and is our ground truth for every decision.
1
3
30
511
Lukas Blübaum retweeted
This morning we released Kolibri. Some details on how we built it in the thread.
16
33
6
468
28,816
Lukas Blübaum retweeted
Stay tuned
2
3
15
1,117
Lukas Blübaum retweeted
Kolibri is out! 78B params, only 3.46B active, Apache 2.0. We taught a hummingbird to fly like a phoenix. 🐦‍🔥
2
5
30
709
Lukas Blübaum retweeted
Kolibri tech report, hope you like this read!
2
19
380
Lukas Blübaum retweeted
Kolibri is out! Had a lot of fun working on this
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
2
32
824
Lukas Blübaum retweeted
Check out what we made! Had so much fun working on this.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
1
23
273
Lukas Blübaum retweeted
Tech report is available: aleph-alpha.com/downloads/te…
18
39
22
671
305,364
Lukas Blübaum retweeted
Focus. Discipline. Delivery. Today: Kolibri. Built end to end by our team in Germany. We also published the full tech report. Methods, results and limitations. 189 pages. To the team: thank you for seeing it through. Made in Germany. Built for the world.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
14
14
1
160
6,327
Lukas Blübaum retweeted
from random weights to the quality-cost pareto frontier in less than two months. a few insights 🧵
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
3
12
123
10,346
Our first model is out! Check out the tech report for the details. Pre-training start to release took less than 2 months.
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
2
3
38
941
Lukas Blübaum retweeted
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
435
1,008
490
9,793
2,695,903
Lukas Blübaum retweeted
The velocity of a team depends on tooling. Our model factory pioneers Model Training as Code, letting us scale our hill-climbing efforts while maintaining acceleration. Read about it below, and stay tuned for more: aleph-alpha.com/en/blog/mode…
13
3
35
6,142
Great deep dive into DSA and the economics of serving long-context models!
deepdive into the economics of DeepSeek Sparse Attention (DSA) and how it affects the profit margins of serving a Claude-Code-like products link in the thread 1/x
1
4
323