Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:

The evolution of interleaved SWA and global layers with window size 512 happened due to merging Routing Transformer (github.com/google-research/g…) with scan layers - XLA likes repeated blocks to reduce compilation time. Doing top-k sparse attention proved to be harder on TPUs and most of the successful sparsification has happened on GPUs (deepseek, minimax etc).

Congrats to @GoogleAI's Gemini 4 Argon for taking the top spot of our Humanity's Sixth Sense (HSS) benchmark with a score of 55.4%, edging out GPT-6-Astra at 53.6%. 🥇 HSS tests AI’s ability to reason beyond what’s visible, from hidden objects to predicting what happens next and understanding human behavior. We took a closer look at performance across categories, and that’s where things get interesting. 👀🧵

Zachary has become my favorite google employee in just 2 dayss.. He was underground for almost a year, probably just bookmarking posts and from the moment Gemini 4 Argon launched.. he has been ruthless in calling out every negative post.. I just hope that when we get hands-on access to the model.. all the call outs stay valid!! Great job @zacharynado and good luck!!

Introducing Gemini 4 Argon, our new frontier model, rolling out to cyber defenders starting today, and more widely as soon as possible. I am really excited by the progress we have made here. Argon is priced at $2 in and $10 out during introductory pricing!

We are hiring research scientists and engineers🧑‍🔬 on the Gemini Omni team! Both in the US (Mountain View / SF) and Europe (London / Zürich). Apply here: google.com/about/careers/app…

Let me take this opportunity to pay tribute to the RLSlow team :) It was a priviledge to lead it through the years, first with @ilyasut then with @merettm and finally on my own - the best team you can ever wish for. It feels like ages ago when we started working on the foundations of reasoning (later berry) with @TrapitBansal, Francis Song, @_smileyball and @sidorszymon. Many more things happened since then, huge thank you to all teammates through the years: Peter Zhokhov, @unsafePerformIO @vladi_petroff @masonmeyer_ @afrozenator @lukasz_kondr @dmdohan @wang12tao @TheRealRPuri @edwardjhu @eqhylxx @suchirbalaji and Dan Selsam for all the help. I hope this great technical work continues to be truly and broadly beneficial. xitter.cf/merettm/status/2096630…

been using flash 3.8 for a few weeks, given its speed and capabilities it's really hard to choose other models for most daily tasks you get the same feeling when you had slow internet back in the 90s and suddenly got access to broadband

Cool eval!

it’s surprising to me how many people seem to not understand that great models are built with super high quality curated data finding novel ways to create / get this data is a huge edge

We’re dropping Gemini Omni: our first step towards a model that can create anything from anything - starting with video. It combines Gemini’s intelligence with our generative media systems - representing a leap forward in world understanding, multimodality, and editing 🧵