AI Research @ ellamind

Joined January 2017
Tokenization Tax Report 2026 now also on @huggingface hf.co/spaces/ellamind/tokeni…
Compared the tokenizers of current models. Efficient: Command A+, Muse (Glimmer and Spark, based on Llama 4’s vocab), Grok 4.5+, Qwen 3.5+, OpenAI o200k (GPT-6, GPT 5, gpt-oss) Claude? +59% tokens for English text...
3
75
Max Idahl retweeted
Replying to @sama
love the ambition and thesis, but given the current tech, i'd say it's your worst product concept so far.
62
48
68
292
Forgot to add: As European AI tradition demands, none of the four European tokenizers we tested beats the o200k baseline averaged across the seven EU languages we experimented with.
Compared the tokenizers of current models. Efficient: Command A+, Muse (Glimmer and Spark, based on Llama 4’s vocab), Grok 4.5+, Qwen 3.5+, OpenAI o200k (GPT-6, GPT 5, gpt-oss) Claude? +59% tokens for English text...
4
150
Compared the tokenizers of current models. Efficient: Command A+, Muse (Glimmer and Spark, based on Llama 4’s vocab), Grok 4.5+, Qwen 3.5+, OpenAI o200k (GPT-6, GPT 5, gpt-oss) Claude? +59% tokens for English text...
3
5
3
17
5,685
Here are the vocab sizes for current models
3
2
133
Forgot to add: follow @magikarp_tokens who is doing amazing tokenizer analysis incl. reproduction of Claude's vocab
1
12
Labs rarely change tokenizers. @OpenAI reached Astra-level capability with a tokenizer they shipped in May 2024 (at least the text part). No need to create your own tokenizer to go frontier. @thinkymachines Inkling also uses the o200k vocab.
1
1
124
we managed to sneak some German flavor into frontier-bench, courtesy of @bla1990so
Forget math proofs, we’re getting to the real frontier now.
1
4
249
if you want more tasks like these, get in touch. we are just getting started
30
iteration is key
very impressive model for the size by poolside, but imo what's even more impressive is the iteration speed 3 (open) models in 3 months 😮
2
327
Tech report on Soofi-S-base is out: A 30B-A3B Mixture-of-Experts Model based on @NVIDIAAI's Nemotron 3 architecture, trained on 27T tokens with a strong focus on German. Supports up to 1M context length. Its a decent model and a solid first step. And SOOFI-L-base is already training. Throughout the project, we executed more than one million benchmark task runs. During the production training run, we continuously evaluated intermediate checkpoints to track whether we are on target. Brief 🧵 on base model evals in SOOFI, a technical blog with more details will follow soon.
📄 Releasing the Soofi S pretraining tech report: a sovereign, open foundation model for German and English Today we’re publishing the full pretraining tech report and project page for Soofi S 30B-A3B — a Mixture-of-Experts hybrid Mamba model trained on ~27 trillion tokens with deliberately up-weighted German. What’s in the report: 🏆 Strongest fully open model in our evaluations on BOTH the English and German aggregates — ahead of Olmo 3 32B and Apertus 70B (full methodology in the report) 📋 Radical transparency: complete per-source data accounting, all hyperparameters, training + eval code, checkpoints — everything under permissive licenses 🇩🇪 Trained end-to-end on Deutsche Telekom’s Industrial AI Cloud in Munich — sovereign AI infrastructure on German soil Soofi S combines frontier-level capability with the highest measured aggregate long-context decode TPS, and unlike full-attention dense baselines maintains high throughput as context grows. The figure plots Capability Index versus measured aggregate decode TPS/GPU at 40K context and batch 32. The Capability Index averages five benchmark groups, i.e., Code, GSM8K, GPQA-Diamond, English aggregate, and German aggregate, after normalizing each group to the best plotted model. Aggregate decode TPS/GPU is measured with a TP=1, one-B200 vLLM latency-subtraction protocol.
5
4
5
28
11,372
The post-training team is working hard to turn SOOFI-S-Base into instruct and reasoning models. The first preview models are already in a private beta with selected partners. Just dont ask me when the model repo will be un-gated for the public, I cant tell.
1
3
289
Shoutout to all project partners for making this happen: @ki_verband @DFKI @FraunhoferIAIS @FraunhoferIIS @TUDarmstadt @Uni_WUE @BHT_bln @l3s_luh @LamarrInstitute @Hessian_AI @MMerantix @ellamindAI and the @BMWE_ for funding the project in the scope of IPCEI-CIS / 8ra. Special thanks to @bla1990so @rscholz_ @bjoern_pl @jphme, not easy to stay sane in eval & looking at data land.
1
5
561