Zachary Charles (@MatharyCharles) header
Zachary Charles (@MatharyCharles) profile picture

distributed machine learning @ google | sometimes mathematician

Seattle
Joined September 2012
Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)Photo by Zachary Charles (@MatharyCharles)
Zachary Charles retweeted
Finally seeing my summer internship project at Google go live as a blog post! 🚀 We implemented Streaming DiLoCo natively in MaxText for distributed LLM training across low-bandwidth, multi-metro networks. Details below 👇 developers.googleblog.com/di…
3
9
2
36
4,880
Zachary Charles retweeted
also can’t discredit that it’s “in vogue” to care about outages and cybersecurity incidents, so of course any outage that happens is going to get a lot of headlines, even if last year the same outage would’ve just made it to the org’s Status page and tech blogs
3
26
4
1,056
55,956
Zachary Charles retweeted
I've tweeted before about how architecture improvements usually improve the constant factor in loss scaling laws, keeping the same exponent. The scaling ladders in the Ripple-65B release from @RicursiveAI are a nice illustration of this at scale! 🧵
5
10
88
5,658
Zachary Charles retweeted
this is cool but it's extremely weird framing imo to call the stacks project "critical infra," that feels like it's importing intuitions from engineering that just don't apply here. if you deleted the stacks project tomorrow it's not like planes would fall out of the sky
Most think vibecoding at scale is slop. Here’s a counterexample. Today we release a 16M-line formalization of the entire Stacks Project, critical infra underlying modern math, with optimized code quality and its highly interconnected structure preserved. A mathematical city 🌆
12
3
1
144
9,014
Zachary Charles retweeted
Hi! this is Rohan Anil, I develop training algorithms among other things - my work trains various frontier models. At CoreAuto, I was fascinated by the problem of training on compute islands, eventually in space training. I hope you all enjoy this write up!🪐🛰️ coreauto.com/hitchhikers-gui…
66
137
39
1,581
193,527
Zachary Charles retweeted
For today's Sunday Review, we're reconnecting with a 2004 cult-classic video game whose postmodernist soundtrack doubles as a treatise on the meaning of life pitchfork.com/reviews/albums…
11
264
96
1,326
211,605
Zachary Charles retweeted
Tinker’s Dimuon is Muon with delayed diagonal conditioning, Polar Express and a rank-dependent update norm. I recovered its update rule through the API, borrowing @mrtnm’s approach to StableReinforce. Here’s how 🧵
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback! See our optimizers guide at tinker-docs.thinkingmachines…
14
31
2
354
20,699
I agree with this and adopted this stance unilaterally on a recent review. I encourage you to adopt it in your reviewing — it’s reasonable to write reviews that say “this is dense unreadable slop, my review isn’t detailed and I recommend rejection solely on that basis.”
Some thoughts on reviewing: spylab.ai/blog/reviewing/ First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%. Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
9
13
6
184
32,786
Zachary Charles retweeted
People really have no idea what string theory is, most have never talked to a string theorist nor bother to read up on it. Its public perception is driven by remarkably confident statements constructing a bogeyman that has nothing to do with the actual research
String theory is what happens when an intelligent system is starved of sensory input and starts hallucinating. Intelligence needs grounding. For this reason, intelligence is practically never the bottleneck to solving impactful problems. The bottleneck is iterated contact with reality.
116
59
11
903
54,762
Zachary Charles retweeted
This website is nothing but anonymous morons.
Replying to @beenwrekt
Insane cope, it's becoming clear why so many of these problems remained unsolved. So-called mathematicians are busy writing bs critique/cope posts like this rather than doing actual math. Lmao.
8
3
127
8,622
Zachary Charles retweeted
Uhhhhh i would have let you install numpy, Claude
43
99
12
5,864
129,452
Zachary Charles retweeted
Wife told me these wouldn’t fit. Little did she know I had trained for this moment for years.
545
20,066
815
360,732
12,021,478
Zachary Charles retweeted
If you want evidence that capitalism is a force for good, just visit a grocery store. My ode to the humble American supermarket, for @WSJFreeEx
1,997
5,271
1,485
39,692
6,600,588
Zachary Charles retweeted
common ml optimization trick: don't show loss curves on wide screens. You model learns slower if you screen is too wide.
34
156
21
3,576
102,344
Why would he make the deficit $1
This quoted post is unavailable.
30
67
1
3,654
97,644
An idea: any paper where all creative mathematical contributions are due to an AI model should be positioned as "an exposition of a proof by Model X". That'd be intellectually honest. It would also add value, rather than DDoS-ing arXiv with daily conjectures. Example below.
I think this kind of rush to put one’s name on duplicative work largely generated by AI is pretty corrosive to the field. Important that we find a way to incentivize work with higher marginal value.
4
14
1
84
11,746
Zachary Charles retweeted
46
1,107
50
30,777
1,404,188
Zachary Charles retweeted
I’ve heard enough, make him president.
12
17
3
445
13,272
Ratioed by Chinese bluegrass fusion
I cannot imagine a Chinese person being emotionally moved by poetry or music
85
2,258
71
29,927
587,724
Zachary Charles retweeted
Genuine LOL
An American interviewer asked Count Binface where he stands on the British political spectrum. 👇
14
532
5
15,669
1,200,602