MIT '25 researching Muon & ML
Palo Alto, CA
Joined January 2023
- Tweets21
- Following17
- Followers858
- Likes121
Pinned Tweet
[1/6] Curious about Muon, but not sure where to start? I wrote a 3-part blog series called “Understanding Muon” designed to get you up to speed—with The Matrix references, annotated source code, and thoughts on where Muon might be going.
Anthropic's heart is gold. Please amplify this message. Please read up more yourself. Especially on a context-compressive platform like X, it's hard to know true and false, right and wrong; but today, it is clear. Godspeed to our warfighters and I hope they get Claude again soon!
A statement on the comments from Secretary of War Pete Hegseth.
anthropic.com/news/statement…
There's more to squeeze out of hardware if you squeeze it really hard. The blogs at the bottom of this thread give creative ideas for what's possible to overlap on GPUs.
[4/6] Chapter 2 is called “Source Code.” This was the original motivation for the whole series: I saw people getting stuck reading through Muon’s code. So I made line-by-line annotations you can hover over to read. No more being confused.
[5/6] Chapter 3 is called “Weight Regulation.” The goal is to orient people toward some exciting recent work on Muon, including @Jianlin_S's MuonClip and our recent paper.
xitter.cf/LakerNewhouse/status/1…
[6/6] Muon is strong because of the community that has pushed it forward. I’m really excited by new work coming out. Let me know if I missed anything. And thanks to all the people who read this blog series and gave early feedback: @jxbz, @jordanjuravsky, @KwangjunA, and others!
[3/6] Chapter 1 is called “Into the Matrix.” Get ready for some fun Neo references while seeing why Muon looks at the gradient as a matrix, not a vector.
[2/6] There’s been a lot of interest in Muon recently, so I wanted to make a practitioner’s guide that's accessible to everyone in the machine learning community.
lakernewhouse.com/muon
[1/9] We created a performant Lipschitz transformer by spectrally regulating the weights—without using activation stability tricks: no layer norm, QK norm, or logit softcapping. We think this may address a “root cause” of unstable training.
[8/9] This is work done alongside fantastic friends and collaborators: @phess002 @leloykun @anzahorodnii @jxbz @phillip_isola
[9/9] We’re really excited where the community can take this.
We’re publishing all our code and data—there’s lots more to test and understand that can help us achieve adversarial robustness, bounded activations, and stable training at scale.
Read the paper: arxiv.org/abs/2507.13338
Code: github.com/Arongil/lipschitz…
Data: huggingface.co/phess2/lipsch…
