Josh Herritz (@jherritz) header
Josh Herritz (@jherritz) profile picture

Obsolete Data Wrangler | AI, data, startups, code, economic development, YIMBY, urbanism, current events and running

Madison, WI
Joined June 2008
Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)Photo by Josh Herritz (@jherritz)
Josh Herritz retweeted
One bullet dodged at least.
27
55
10
908
57,689
Josh Herritz retweeted
as some of u may have suspected, i have recently joined @METR_Evals :) this is one of the first things i worked on there, along w the rest of the capabilities team, very excited it's finally out. understanding what the hell is going on with models and especially where their remaining gaps are is incredibly critical for informing our decision making as a species. there's a shitload of stuff in here, really excited for people to wade into the appendices, and i've even made sure to make all the backing data behind charts easy to download so ppl don't have to do pixel peeping to claude up their own charts. some random (personal) takeaways: - it would be really sick if time horizons continued to work as an easily interpretable measure of capability... but unfortunately i think that's just no longer true. there's just too much variance in the spikes. - as much as i'd love for IRT methods to result in *intuitive* understanding they're just kinda structurally very uninterpretable, but remain basically our best way to get ~reasonably robust aggregate info - swarms are complicated! model behavior in swarms is super sensitive to all kinds of factors, inter-run variance is high, "swarm cultures" can form and significantly affect performance. for some things, clear strong uplift! for others, swarms are a downlift. love it when u get results that are basically "haha good luck making a trend outta this" - getting humans to outperform models is really rough now. really extremely rough. even with some model assistance, for basically everything verifiable we're pretty much at the point where only ~world class experts still beat models, and it's hard af to get world class experts to spend two straight months working on a METR task - it's pretty crazy watching models advance past what you thought was potentially possible on a task, and then go further, and then further, until you're just no longer sure where the upper limit actually is somewhat more personally, on the Tenobrus account. historically i feel like i've actually done a lot of good for safety by being a "brawler", wading into fights, saying things i thought were true regardless of consequences, saying shit that i just kinda thought was funny at the time... for a long long time, my account has reflected on no one but me. but i don't think i can really remain in that place while at METR. it would be disingenuous to remain anon and speak without disclosing my affiliations while working for the transparency organization. it's 2026, it's crunch time, it's time to step up and try to provide the kind of independent and carefully truthful institutions the world needs rn. so my style here is gonna change. that's okay, it's already been changing for a long time! and METR is worth it, i think it's one of the single most impactful places in the world to be right now, and i'm not going to fuck up that responsibility. let's keep figuring this shit out :)
We've been working on understanding model capabilities beyond what our prior time horizon tasks could capture. Today we’re sharing a research update. We find that AI R&D-relevant capabilities are improving rapidly and are getting harder to measure.
151
40
2
1,312
27,293
Josh Herritz retweeted
This is the same unusual and unexpected jump in capabilities that Anthropic observed in Fable 5.2. This sudden improvement was so significant in GPT-6.2 that OpenAI restructured their dedicated robotics team.
GPT-6.2 for robotics:
24
35
4
593
23,628
Josh Herritz retweeted
*the open-source loopers are coming it's so over
Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index. thinkingmachines.ai/news/ink…
14
10
1
343
13,244
Josh Herritz retweeted
Humanity built these AI systems and it is humanity that should be proud of the achievements we accomplish with them. Today we should be proud of the stellar folks at Ant and @__alpoge__ in particular for showing us the way past the frontier.
7
16
2
246
10,213
Forbes just ranked Culver’s #1 on its ‘America’s Favorite Restaurants’ list for 2027
232
1,413
1,044
18,559
1,092,047
Josh Herritz retweeted
It is becoming clearer that Secretary Lutnick and Commerce Department disregarded national interest, national security, and AI competition with China against all advice, warnings, and classified intelligence. Hard to compete when we just give China the answers. Eyes wide open. @ChinaSelect @WSJ
2
50
3
161
2,484
Josh Herritz retweeted
Between this and The Altruists, I feel like Hollywood does not understand the Bay, does not want to understand the Bay, and will pump out generic 'nerds suck' movies regardless of what complex villainy and hubris and ideological, catastrophic recklessness they ostensibly depict
New trailer for Luca Guadagnino's ‘ARTIFICIAL’, starring Andrew Garfield as Sam Altman. The film follows the story involving OpenAI focused on the firing & rehiring of CEO Sam Altman. In theaters on January 8.
69
47
19
1,077
45,240
Josh Herritz retweeted
There is a lot of great stuff in this new report from the California government on AI policy. Particularly valuable to have mechanisms where the government can modulate the speed of development, especially for recursive self-improvement. govops.ca.gov/wp-content/upl…
12
32
6
195
19,237
Is anyone in government anywhere capable of operating at "in a few months" urgency levels?
Replying to @janleike
There are a lot of details you could quibble with, e.g. I'm not sure "compute devoted to automated AI R&D" is the best metric to look at. But we don't have much time to quibble since we might already want to actively modulate the speed of AI development in a few months.
53
Josh Herritz retweeted
New: In a classified communication to DNI Jay Clayton, Democratic Rep. Jim Himes -- the incoming chairman of House Intelligence Committee -- made clear lawmakers will restrict funding for SpaceX systems unless Trump admin & Musk provide answers on slowdown of Ukraine’s Starshield connection speed, per an excerpt obtained by CNN. cnn.com/2026/11/20/politics/…
110
869
64
2,529
87,204
Josh Herritz retweeted
Detaining a PhD student on her way back to Princeton from presenting scientific work at a conference is not how we get the best and brightest scientists in the world to want to come to the US. I hope this is resolved quickly! More details: dailyprincetonian.com/articl…
Our colleague Eva Yi Xie, a Princeton neuroscience PhD student, was detained by ICE at Reagan National Airport on her way home from SfN, where she’d just presented her poster. She’s being held in Chantilly, VA. Please help us bring her home. 📷
271
565
33
5,930
336,994
Josh Herritz retweeted
We have a full confirmation from Andrew Sutherland who first managed to finish the nanoda kernel run and comparator in paranoid mode. This is now basically saying that the Riemann Hypothesis is 99,99999% true. It's almost impossible that any glitch has been deliberately implemented into the code and that anything is missing from the proof artifact (no sorries, no additional axioms, no weird bugs).
We don't yet have full confirmation from the paranoid-mode comparator test, but we are close to seeing a formal artifact generated in Lean that provides a proof of the famous Riemann Hypothesis. This is such a historic moment. The code was generated by @__alpoge__ using Claude and possibly other software. There is still some uncertainty, but we may be on the verge of a historic announcement. For the first time in history, we may have such a magnificent proof that we DON'T fully understand yet, but that is probably correct.
53
317
60
2,504
264,878
Josh Herritz retweeted
Oracle and Vantage are running a $15 billion data center in Port Washington, and just announced they are donating $27 million for a new community center in the city. I like this quote from Port Washington's mayor on how that conversation went.
16
11
3
68
6,910
Josh Herritz retweeted
44
217
16
4,000
158,975
He's not wrong! Mathematicians should be rushing to work with and learn from @__alpoge__.
Replying to @ObhishekSaha
Also, I wish people would stop calling it the "Claude proof", as I am seeing on this website, and instead credit the person involved: Levent Alpoge. Levent is not just an Anthropic employee, he is a top-level mathematician. Others with similar access to frontier AI models/tokens (and trust me, there are many such people) were not able to come up with a proof of RH. Levent has been on a truly generational run since summer, proving time and again his ability to solve conjectures thought out of reach. Jacobian conjecture, Hadamard matrices, complex structure on S6, etc. And now GRH. I think the mathematical community, a large part of which is anti-AI, has treated him poorly, at least from what I've seen on social media. The honour of proving RH, if the proof holds up, belongs to him.
1
3
145
Josh Herritz retweeted
Also, I wish people would stop calling it the "Claude proof", as I am seeing on this website, and instead credit the person involved: Levent Alpoge. Levent is not just an Anthropic employee, he is a top-level mathematician. Others with similar access to frontier AI models/tokens (and trust me, there are many such people) were not able to come up with a proof of RH. Levent has been on a truly generational run since summer, proving time and again his ability to solve conjectures thought out of reach. Jacobian conjecture, Hadamard matrices, complex structure on S6, etc. And now GRH. I think the mathematical community, a large part of which is anti-AI, has treated him poorly, at least from what I've seen on social media. The honour of proving RH, if the proof holds up, belongs to him.
29
70
13
1,014
47,249
Josh Herritz retweeted
Like most mathematicians, I believed the Riemann Hypothesis to be completely out of reach at present (even with the best AI tools), because I expected it to require substantial theory-building, probably in arithmetic geometry. The kind of new theory-building that fills entire books, and that AI isn't so good at yet. Mathematicians have a notion of how deep a problem is. A sufficiently deep problem is generally intractable until enough of the theory is in place. If difficult problems are like valuable mineral reserves buried miles underground, you first need to build the infrastructure to reach them. What Levent's proof seems to have done instead is something completely different. It takes an incredibly complicated tour through the infrastructure built for a hundred other mineral reserves, situated far from the destination, and somehow create a wormhole to the original destination. This is not theory-building in the usual sense. It is something much weirder and more alien. The best human mathematicians know how to build theory, painstakingly, to reach a goal. But no human brain, on its own, is capable of creating a wormhole that requires a 25,000 page long tour through existing infrastructure with no obvious or realistic connection to the goal. But huge credit to Levent here: I think no current AI, on its own, is capable of this either. I had never worked on or thought too much about the Riemann Hypothesis: it seemed very far removed from my own area of number theory. But the astonishing thing about Levent's claimed proof of RH (which I think has a very good chance of being correct) is that it uses a huge number of precisely the things that are central to my own corner of number theory: - Theta functions and other automorphic forms on unitary groups - Gan–Gross–Prasad period formulas - Relative trace formulas Etc., etc. A problem that I had always assumed would require new theories in arithmetic geometry may instead turn out to be tractable through an inhuman chain of existing (deep) results in higher-rank automorphic forms (and a lot else).
This actually makes me a bit more confident that the proof has a good chance. RH is proved by using *everything*!
121
481
98
5,006
479,787
Josh Herritz retweeted
A good time to step outside, take a walk, touch some grass, and ruminate over this for a bit
14
86
11
1,262
20,848
Josh Herritz retweeted
This Masullo RH story isn't looking like it's going to hold up
Replying to @ElliotGlazer
Followup: Fails an Astra audit. Come on what were these guys doing the past few months🤦‍♂️
20
11
6
443
33,437