Rex St. John

Meta published a paper that might end the current LLM era. They proved you don’t need tokenizers to train SOTA models. Every major AI (ChatGPT, Claude, Llama) reads text in "tokens", chunks of words. This requires massive vocabularies, usually around 100,000 different tokens. It's how models have always been built. But researchers at Meta proved this is a massive bottleneck. They trained a model to read raw "bytes" instead. Only 256 basic units. The plot twist: They took a tiny 1-billion parameter model and fed it up to 1 trillion bytes of data. At first, the standard token-based model was winning in the low-compute regime. But then something crazy happened. The token model hit a ceiling. It plateaued. It completely stopped getting smarter. The byte model just kept climbing. By converting token probabilities into pure byte-level reasoning (using a new distillation method called "End-Of-Token"), the tiny byte model shattered the performance ceiling. It matched the standard token model using just 1/6th of the training data. It crushed storage costs by 80%. And asymptotically, this tiny 1B-parameter byte model outperforms Llama 3.2 and even the 2-billion parameter Gemma model across the board. Nobody is talking about the real implication here. We have spent years forcing AI to read text the way humans do. It turns out, AI gets much smarter, much faster, when you let it read like a machine.