Research Engineer @ Google DeepMind https://xitter.cf/t.co/h1NwL23Yfh
London
Joined June 2022
- Tweets14
- Following34
- Followers179
- Likes111
Pinned Tweet
Sure matplotlib is cool, but what if I want to load my loss curves into the 2006 hit Flash game LineRider?
Josef Dean retweeted
The DiLoCo team at Google DeepMind and Google Research is proud to release Decoupled DiLoCo, the next frontier for resilient AI pre-training.
Decoupled DiLoCo enables training with datacenters across the world, using heterogeneous hardware, and never halting the system despite hardware failures.
Josef Dean retweeted
Our Papers of the Month for August is now live! This time we're digging in to:
Spectra, Scaling LLM Test-Time Compute, and Training Language Models on the Knowledge Graph 🧵
graphcore-research.github.io…
Data from the u-μP paper arxiv.org/abs/2407.17465
Code at github.com/jndean/LossRider
Music generated using Udio.
Maps loaded into linerider.com
Josef Dean retweeted
Our u-µP paper hit arXiv this morning! I'm so proud of this one — and grateful for a wonderful team who put so much into it 🥰
We add lots of good things to µP. Better sweeping, transfer, simple FP8. Already @cloneofsimo has a great thread on it, which I highly recommend
babe wake up, new muP paper dropped
arxiv.org/abs/2407.17465
And holy smokes does this look promising!
