Lukas Blübaum retweeted
Expert load balancing is key to training Kolibri efficiently at scale.
We developed two new methods: Exact Quantile Balancing (EQB) for global balance and Load-Error Injection (LEI) for local balance. @JesseLiJiaze and @PitNeitemeier explain them in a new, visual blog post.
Introducing North 2.
Enterprises have been compromising on agentic AI for years: platforms that aren't scalable, models that aren't sovereign, control without enterprise-grade security.
That ends today, with 15+ new features in our biggest upgrade yet:
Lukas Blübaum retweeted
My favorite reinforcement learning plot from our Kolibri report: Inference v.s. training log probs, background is the logit gradient norm of importance-sampled Reinforce. Numerically unstable for small inference log-probs, but you can do exact token-level gradient clipping in RL.
Lukas Blübaum retweeted
New Apache 2.0 model from aleph alpha. It rocks. This is the way. So excited to build together.
New Apache 2.0 model from @Aleph__Alpha. It's really good.
If you like that, you're going to love what we build together.
Lukas Blübaum retweeted
Proud of what we achieved together. We’ve shipped a good model, but also built an incredible team that’s moving at high velocity. Kolibri climbs faster than Kolibri Origin. Excited for what’s next.
Replying to @Aleph__Alpha
Kolibri went from the start of pre-training to taking flight today in less than two months. We built the foundations from January. From the beginning, we focused on fast iteration and on building a pipeline for the long term. This was enabled by our Model Factory, which orchestrates the Kolibri training and evaluation, and is our ground truth for every decision.
Lukas Blübaum retweeted
This morning we released Kolibri. Some details on how we built it in the thread.
Lukas Blübaum retweeted
Kolibri is out!
78B params, only 3.46B active, Apache 2.0.
We taught a hummingbird to fly like a phoenix. 🐦🔥
Lukas Blübaum retweeted
Kolibri tech report, hope you like this read!
Lukas Blübaum retweeted
Tech report is available: aleph-alpha.com/downloads/te…
Lukas Blübaum retweeted
Focus. Discipline. Delivery.
Today: Kolibri. Built end to end by our team in Germany.
We also published the full tech report. Methods, results and limitations. 189 pages.
To the team: thank you for seeing it through.
Made in Germany. Built for the world.
Lukas Blübaum retweeted
from random weights to the quality-cost pareto frontier in less than two months.
a few insights 🧵
Our first model is out! Check out the tech report for the details. Pre-training start to release took less than 2 months.
Lukas Blübaum retweeted
Small bird, fast wings, Kolibri is here.
78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe.
Now the weights are yours. Run it on your own hardware, under Apache 2.0.
Lukas Blübaum retweeted
The velocity of a team depends on tooling. Our model factory pioneers Model Training as Code, letting us scale our hill-climbing efforts while maintaining acceleration. Read about it below, and stay tuned for more: aleph-alpha.com/en/blog/mode…






















