Joe Davison (@joeddav) header
Joe Davison (@joeddav) profile picture

AI research & engineering. Prev: zero shot tooling @huggingface, structure elucidation @enveda

Lehi, UT šŸ”
Joined January 2009
My first day at @huggingface, weeks before COVID hit NYC I was only there for about 18 months, but I'm incredibly grateful to have worked with these folks šŸ¤— Congrats to @ClementDelangue, @julien_c, @Thom_Wolf, and everyone who helped turn HF into the leaders of open AI!
4
1
58
5,075
AlexNet was closer to the launch of Napster than to today
The first episode of the Simpsons was closer to the JFK assassination than to today
4
308
as I often tell my wife: > we just have to make it through one more work week > and then one more weekend > and then one more work week > and then one more weekend > and then one
parents of young kids, with jobs... when do you rest? I get to the end of the weekend and I'm more tired than on Friday
2
287
Replying to @a_musingcat
slop means it takes less human effort to produce than it does to consume (??)
1
2
10
808
non-EA’s sneaking around Anthropic HQ with their IPO loot:
There were so many non-EA guys at Anthropic who snuck through the interview. Just silently praying for the IPO of a lifetime while their whole company crashes out and declares imminent apocalypse.
1
2
665
Joe Davison retweeted
Sometimes I just think about it... 🄹
26
11
4
432
18,816
Joe Davison retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
135
794
153
6,571
841,681
Joe Davison retweeted
Replying to @scaling01
Yeah I’d say the RL is working now all right
You can tell the RL is done properly when the models cease to speak English in their chain of thought
104
189
34
5,213
295,726
GLM-5.3 reasons that a misaligned action taken during training could be reinforced and make similar actions more likely later. It also considers what kind of agent it wants to be in the future, and factors this into how it should act. We call this "update-aware reasoning".
4
24
25
355
20,378
Joe Davison retweeted
Introducing ChatGPT for Dishwashing GPT-6 achieves state-of-the-art results on dishwashing benchmarks for long-running jobs and difficult stains. The Navier–Stokes Millennium Prize Problem is a $1 million challenge about whether the equations describing fluid motion can break down. In this video, I accidentally found a counter-example while trying to optimize my dishwasher. But in reality, OpenAI used 10,000 AI agents to find a proof in 88 hours: openai.com/index/navier-stok… Parody of the original GPT-6 Astra ad @OpenAI #ChatGPT_Partner #chatgpt Credits: Written and Directed by Joma Co-Writer - Henry Connor Coan IV @coan_henry Production by Dauntlus Studios @dauntlus.studios Producer/Co-Director - Marcus Liew @dauntlus Camera Operator - Amos Lee @amoselijahlee Art Director - Alethea Soo @aletheasoo Art Assistant - Alicia Lim @aliciatkl_ Assistant Editor - GPT-6 Astra Music: Tensions Run High by Soundridemusic youtube.com/watch?v=Ly9H63SL…
423
960
342
10,862
1,145,361
Joe Davison retweeted
Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @balesni and @j_asminewang had been fired too. Why did OpenAI suddenly stop trusting us? This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them. I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job. For months, I’d been raising safety concerns thatĀ we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired. I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership. I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
229
1,140
176
5,959
658,759
Joe Davison retweeted
Ten years of Sad Affleck memes and it turns out he'd just been trying to install CUDA drivers the whole time.
58
1,153
67
15,414
271,854
Joe Davison retweeted
No hype, just the results. Good.
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math
7
5
217
18,103
pre-training is now just a warmup phase
Post-training / RL and inference is taking over compute
1
1
5
503
How fast is AI's research taste improving? We find that the experimental research taste of frontier models has doubled every ~3 months since December 2025. The best model, Opus 5.5, now exceeds our expert human baseline. Our human experts are experienced researchers, but most haven’t worked at a frontier lab. Why measure research taste? In the AI Futures Model, it largely determines how quickly artificial superintelligence is reached once coding is fully automated.
58
206
104
1,617
303,137
Who called it losing chain of thought monitorability through opaque internal reasoning and not ā€œweight, weight, don’t tell meā€
21
37
1
388
11,544
Joe Davison retweeted
Replying to @mervenoyann
IMO it’s a better name. ā€œZero shotā€ was a useful term in 2020 because the default way of doing classification was to train a bespoke model. Nowadays almost everything is ā€œzero shotā€, so the term doesn’t have the same discriminate (pun intended) power.
1
6
392
Joe Davison retweeted
Today we're unveiling Trillium Labs @trillium_labs, a new non-profit to foster the open science of frontier AI. We're building open post-training recipes and will expand into open infra to study RSI, reward-hacking, multi-agent systems, and whatever comes next. We're built around the theory of change that you need more eyes to solve hard technical problems. We have faith in the scientific methods and communities that humanity has built, and worry that AI is becoming too closed to utilize them. Trilliums are wildflowers that bloom briefly in the spring, before the forest canopies fill out. Though they are small, they lay the foundation for the cycles of growth and nourishment through the rest of the year. At Trillium Labs, the recipes will be the slow nutrients for the seasons and the model releases will be the blooms. Building an institution dedicated to this is needed because, much as nature’s trilliums are slow to expand and grow, the open-ecosystem needs time and dedicated resources to catch up. I co-founded with with a long-time friend and collaborator Tom Zick (@thesezickbeats). We're hiring (full time + student collabs/interns), we're fundraising, and we're looking for compute. Please get in touch if you're interested in helping out. Offices based in the Bay Area and Cambridge MA, remote okay. I’m in the Bay Area until for The Curve and COLM to connect with people who are interested. We’re thankful to have initial support from Halcyon Futures and Schmidt Sciences with more funding en route to enable our ambitions of scaling. Our advisors @Thom_Wolf, @HannaHajishirzi, @gneubig and @ctnzr have been instrumental to building the ecosystem that exists today, and I’m stoked to get to keep working with them.
250
301
83
3,280
261,774
Joe Davison retweeted
Ezra Klein asked Bill Gates whether normal corporate incentives are enough to handle AI risks. Gates doesn’t mince words:
...very interesting & somewhat troubling deep dive look at AI, its potential dangers without control and its potential benefits. Keeping AI out of the hands of people with bad intentions will be the big challenge. nytimes.com/2026/09/29/opini…
127
657
75
4,938
573,854
CoT monitorability:
We think computational depth is the missing scaling axis, i.e. we should be doing a lot more deep learning! Every other axis has been scaled by OOMs over the past few years (params, data, sparsity, test-time reasoning), but depth has been stuck at ~100 layers since GPT-3. We've found that LLMs are both *severely* depth-bottlenecked and bad at using the depth they have, and that architectural interventions that lift this bottleneck efficiently lead to gains that increase with compute. w/ @akshayvegesna
1
7
515
Joe Davison retweeted
Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out. It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend. The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans). More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains. In Machines of Loving Grace, I wrote about AI’s potential to ā€œcure most diseases in 5-10 yearsā€ — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries. Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and developing them into powerful tools for biotechnology. I’m proud of the resources Anthropic has invested in accelerating the public benefits of AI through the life sciences, and we’re aiming both to grow our life sciences team and to work with other scientists to extend this approach to a broad range of problems. If you have a proposal for a research collaboration or are interested in joining our life sciences team, please reach out.
Claude has discovered a previously unknown enzyme system hidden in the DNA of bacteriophages. Beside the enzyme’s gene sits a long array of repeating DNA—a structure that looks somewhat similar to CRISPR. We don’t yet understand what this system does, but only a handful of known systems share its features, and all of them are able to cut, copy, and paste DNA. Historically, the discovery of such programmable systems has helped revolutionize medicine. CRISPR, for instance, is now the foundation of genetic medicines. But it will take much more work to learn what this system does, and whether it can be put to similar use. Read more: anthropic.com/news/claude-di…
1,682
3,428
964
29,973
5,625,219