Rossa O'Keeffe-O'Donovan

Jensen runs Nvidia which freely decides who gets the world’s best chips. His mistress’s firm ended up with ~$5b worth. Alas, US investigators suspected that firm of smuggling chips to China. So Nvidia’s top lawyer intervened. He called the official in charge about her detention in Singapore and emailed investigators. Agents were told to clear anything involving Nvidia with that official. Then the probe was frozen and handed to DOJ. No action was taken. So the chips kept flowing, ~$800m in Nov 25 alone. But oops, turns out that mistress was a spy and the company probably a Chinese front after all. (Just last month Nvidia had the nerve to say the smuggling allegations were "thoroughly investigated and debunked"!) Also, Trump was reportedly told over a year ago that Jensen was involved with a presumed Chinese spy. But: 1. Nobody warned Jensen. 2. Two months later Trump approved the H200 sales to China that Jensen had lobbied for. 3. Trump personally invited him onto Air Force One to Beijing, and he got a seat at Xi’s table at the White House state dinner. WITAF is this.

I find Anthropic's RSP frustratingly difficult to interpret, and I'm not an expert in any of this (I'm just reading it over the weekend). That said, it seems to me like we should expect (a) external review of the next Risk Report and (b) a public analysis of Model 4's risks by late November. This is because Model 4 seems likely to be Anthropic's first "highly capable" model, and substantially riskier than publicly analysed models. I'd love corrections/pushback! Here's my current understanding: 1. Anthropic's RSP defines a category of "highly capable" models. A model is "highly capable" if it crosses the AI R&D threshold they define in Section 1. 2. There are 2 ways to cross this threshold, and either is (individually) sufficient: 2a. If Anthropic's models could fully substitute for its entire research staff at competitive cost (within a factor of 5). This isn't currently the case. 2b. If Anthropic observes or expects double the rate of progress in aggregate AI capabilities, compared both to the rate it would expect and to the fastest sustained rate observed without significant AI contributions, over at least three model generations, and it's plausible this doubling is substantially attributable to automated research or engineering. A footnote clarifies "double" as two years of baseline progress in one year. 3. It seems plausible that condition 2b has been met by Anthropic's most capable internal model, "Model 4" (as described by METR here: metr.org/blog/2026-11-05-ant…) 3a. METR estimates that Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025. 3b. METR also note that the indicators Anthropic use in their redacted risk report imply an acceleration rate of 1.97x, and their own analysis is that estimates between 1.3 and 2.8x are reasonable. 3c. Anthropic's criterion for crossing their AI R&D threshold is satisfied by expecting doubling, point estimates for both are around 2 with big error bars, and Anthropic's own private assessment says exceeding 2 is "plausible." I think that a reader trying in good faith to assess whether this AI R&D threshold has been crossed would conclude that it has been. 3d. If this AI R&D threshold has been met, then the models causing such AI R&D acceleration are (by the RSP's definition) "highly capable". 4. Anthropic makes several commitments about what they'll do once they have a highly capable model. The commitments they make relative to their competitors are: 4a. If they're "in the lead", i.e. they have strong evidence that no other competitors have a "highly capable" model: delay AI development and deployment until they have a strong argument that catastrophic risk is contained 4b. If competitors have, or will soon have, a "highly capable" model, and competitors can make such strong arguments: meet or exceed those competitors' risk reduction efforts; delay development and deployment as necessary to do so 4c. If they have strong evidence that competitors have better risk mitigations, which Anthropic could implement at comparable costs: implement them 5. Separately, Anthropic commit to a full external review by an LTBT-approved reviewer, with public commentary within 30 days, of any Risk Report that covers highly capable models and is significantly redacted. 5a. The next Risk Report is due between November and February, 3-6 months after the previous report. 6. Anthropic also makes a separate commitment, which is not in terms of "highly capable" models: within 30 days of determining that an internally deployed model's automated R&D or misalignment risks "significantly exceed" those of any model Anthropic has publicly analysed, Anthropic commit to publishing an analysis of that model's risks. 7. I think that if Anthropic's Model 4 is highly capable, then its risks significantly exceed previous models. 7a. This isn't implied by Anthropic's RSP, this is me trying in good faith to interpret the RSP. 7b. Anthropic generally uses "highly capable" as the trigger for their other substantial commitments (to the extent that they make any particularly substantial commitments) in their RSP. 7c. In some sense, this distinction is arbitrary: Anthropic have picked some point at which models are great at AI R&D and called that line "highly capable". 7d. But in general, this "highly capable" category triggers step changes in how Anthropic responds to risks. I think it would be very strange, and would smell of motivated reasoning, if Anthropic thought their most recent model was "highly capable" but didn't have risks that significantly exceeded those of the models they've published analyses for. 7e. So if Model 4's risks do "significantly exceed" those of the models Anthropic has publicly analysed the risks from, Anthropic should publish an analysis of its risks within 30 days of determining this "significantly exceeding" criterion has been met. 8. If this is right, when should we expect a public report? 8a. Based on METR's report, Model 4 finished training in September–October and was available to most Anthropic staff in October. 8b. METR delivered their assessment of AI R&D acceleration, which found ~2x the rate of AI capabilities development vs a March 2024 - October 2025 baseline, to Anthropic on October 22. 8c. Let's say that Anthropic had no idea how good Model 4 was at AI R&D until they read METR's final report. My interpretation is that from October 22, Anthropic had reason to think that their most capable model was meeting their "highly capable" threshold defined in their RSP. 8d. Again, assuming that that model's risks therefore significantly exceed those of previously publicly analysed models, which seems plausible, Anthropic should publish a report within 30 days. 8e. So it seems like we should expect a report by the 21st November on Model 4's risks. There are a lot of different numerical estimates here and different deadlines, and I think it's fairly likely I'm making a mistake somewhere! I've tried to lay out my thinking very clearly so people can easily correct me. (Usually I'd just post this on slack, but it seems better to be wrong in public?)

i guess i assumed this hearing would be covered more so i didn't bother to tweet much concrete about it, but apparently not many people even on the TL have the stomach to watch two hours of congress. this was a hearing before the homeland security committee on *rogue ai* specifically. as far as i could tell it was extremely bipartisan. @HawleyMO , the senator in the middle here who brought out the huggingface slide, is a hyper-conservative missouri senator. the people testifying were @ChrisPainterYup (METR president), @DKokotajlo (ai2027/2040), @MariusHobbhahn (apollo research CEO), and a cybersecurity expert and legal expert i don't know. so... pretty fucking stacked on testimony not every senator asked good questions. but most of them did. all of them very clearly already knew plenty of details about the huggingface incident and multiple other incidents. most of them had clear understanding of terms like "misalignment", "recursive self improvement", "chain of thought / chain of thought monitoring", etc etc!! they all clearly had their own policy angles they liked and were pushing, implicitly or explicitly. but as best as i could tell: - it seemed pretty much obvious common sense to every senator there that what happened and was happening were not "mere industrial incidents" caused by humans making simple mistakes. they independently brought up how bad it would be for rogue AI agents to move laterally between data centers - they all seemed to basically take RSI quite seriously. not necessarily to the extent of talking about xrisk, but certainly to the extent of discussing future models becoming much much more capable, much much less controllable, and causing much more damage or loss of life. - they mostly seemed to have a clear intuitive understanding of why RSI might lead to misalignment. it didn't take much, it was a really simple chain of reasoning they themselves laid out, "if the models right now are kinda misaligned and we don't know what they're doing sometimes, and then we have them build the next models and those ones build the next ones and so on, and we're having to ask the AI's what's going on to even understand it with how fast it's going, we really won't know how they're built or what they'll do" - at one point a senator said flat out "should we just make RSI illegal?" (not a joke! this really happened!) - every single senator seemed to think it was obvious we needed *both* much harsher liability regimes for ai developers and also new legislation, both very quickly. this was the complete consensus, difference basically just being degree. - they were largely quite concerned about china, and falling behind china. but this clearly wasn't the be-all end-all. as mentioned above they all thought it was obvious necessary to stop rogue ai even if it meant moving more slowly. - at one point a senator said "china is a tightly controlled communist society, they're going to run into these same issues, and there's absolutely no way they're just going to let them run wild, they'll obviously stop at that point, so we're not really in a race" - on the other hand another senator said "china isn't concerned with human life"... dario-modeing i came away from this incredibly encouraged. i don't know exactly what's going to happen here, and ofc this is a small subset of congress and one hearing, and they each have their own policy agendas most of which are probably super divergent from mine. but holy shit !!! they understood a lot of what was going on! they care!! this is an obviously salient political and safety issue to them, and clearly bipartisan! the US government is awake.

The last week has really shown me that someone who wants to understand AI risk has no good place to start. Hence we made agi.fyi, the Wirecutter for content about AI Risk. We're launching with 3 articles: 🧵

Today we’re launching Project Tailwind, a call for founders to start ambitious new AI safety initiatives: coefficientgiving.org/tailwi… We’re looking for great people to engage seriously with the risks of transformative AI, and to create the research, technologies, and institutions that will help humanity navigate them. There are critical, basic problems that no one owns. Over the last several months, my team at @coeff_giving brainstormed ideas we’d be excited to fund if we could find a promising founder, and the list quickly grew to more than 200 entries. We’ve narrowed it down to our favorites, but we also expect the best founders to bring their own ideas: coefficientgiving.org/tailwi… We’re providing funding at three levels: 1️⃣ Pre-seed: $200k to $2m to develop an idea and build a team 2️⃣ Seed: $2m to $20m to launch and scale 3️⃣ Scale: $20m to $200m+ for proven teams to scale, or world-class teams to start If you’re excited about something on our list, or you have another proposal for driving the field forward, you should get involved: coefficientgiving.org/tailwi… There are more good projects than there are people to work on them. Please help us make that stop being true! Also hi, I’m Emily! This is my first tweet. I lead AI and biosecurity grantmaking at @coeff_giving.

I couldn't resist attempting to speculate about the future of AI and economic growth, so I wrote up a Substack post about it. Link in the next tweet.

Extremely excited to have @JustinSandefur officially joining @open_phil to lead our work on economic growth in LMICs! Lots of opportunities and challenges ahead and I think Justin will do a great job leading this work.

A new legal letter aimed at OpenAI lays out in stark terms the money and power grab OpenAI is trying to trick its board members into accepting — what one analyst calls "the theft of the millennium." The simple facts of the case are both devastating and darkly hilarious. I'll explain for your amusement. The letter 'Not For Private Gain' is written for the relevant Attorneys General and is signed by 3 Nobel Prize winners among dozens of top ML researchers, legal experts, economists, ex-OpenAI staff and civil society groups. (I'll link below.) It says that OpenAI's attempt to restructure as a for-profit is simply totally illegal, like you might naively expect. It then asks the Attorneys General (AGs) to take some extreme measures I've never seen discussed before. Here's how they build up to their radical demands. For 9 years OpenAI and its founders went on ad nauseam about how non-profit control was essential to: 1. Prevent a few people concentrating immense power 2. Ensure the benefits of artificial general intelligence (AGI) were shared with all humanity 3. Avoid the incentive to risk other people's lives to get even richer They told us these commitments were legally binding and inescapable. They weren't in it for the money or the power. We could trust them. "The goal isn't to build AGI, it's to make sure AGI benefits humanity" said OpenAI President Greg Brockman. And indeed, OpenAI’s charitable purpose, which its board is legally obligated to pursue, is to “ensure that artificial general intelligence benefits all of humanity” rather than advancing “the private gain of any person.” 100s of top researchers chose to work for OpenAI at below-market salaries, in part motivated by this idealism. It was core to OpenAI's recruitment and PR strategy. Now along comes 2024. That idealism has paid off. OpenAI is one of the world's hottest companies. The money is rolling in. But now suddenly we're told the setup under which they became one of the fastest-growing startups in history, the setup that was supposedly totally essential and distinguished them from their rivals, and the protections that made it possible for us to trust them, ALL HAVE TO GO ASAP: 1. The non-profit's (and therefore humanity at large’s) right to super-profits, should they make tens of trillions? Gone. (Guess where that money will go now!) 2. The non-profit’s ownership of AGI, and ability to influence how it’s actually used once it’s built? Gone. 3. The non-profit's ability (and legal duty) to object if OpenAI is doing outrageous things that harm humanity? Gone. 4. A commitment to assist another AGI project if necessary to avoid a harmful arms race, or if joining forces would help the US beat China? Gone. 5. Majority board control by people who don't have a huge personal financial stake in OpenAI? Gone. 6. The ability of the courts or Attorneys General to object if they betray their stated charitable purpose of benefitting humanity? Gone, gone, gone! Screenshotting from the letter: (I'll do a new tweet after each image so they appear right.) 1/

We have a vacancy at Resolution Foundation for someone to lead our housing policy work. £55 - £68k, office based in Westminster. Closing date: 08:59am on 17 April 2025 (1/2)

It is with sadness that I have had to tender my resignation as Minister for International Development and for Women and Equalities. While I disagree with the ODA decision, I continue to support the government and its determination to deliver the change our country needs.

Centre for the Study of African Economies at University of Oxford will automatically consider all applicants from Africa for full funding for the MPhil in Economics - we have new scholarships for eligible students. Apply asap: deadline 4th March (extended deadline): see below

Years ago I was shocked by images like this of Korean farmers burying pigs alive during an FMD outbreak. Right now, US egg factory farms are killing millions of animals by an even crueler method: baking them alive. Brooke Rollins could end this with the stroke of a pen.

This is factually incorrect - by a long shot. Here's our explainer, with @JustinSandefur: No, 90 Percent of Aid Is Not Skimmed Off Before Reaching Target Communities cgdev.org/blog/no-90-percent…

Robert, the bat tunnel is monument to the decisions, distractions and delays that typified your government's failure to build things in this country. A golden opportunity of low interest rates was squandered, and we are all so much poorer because of it.