Richard Ren (@notRichardRen) header
Richard Ren (@notRichardRen) profile picture

Working on catastrophic AI risks. Research scientist & engineer @CAIS

San Francisco, CA
Joined January 2023
Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)Photo by Richard Ren (@notRichardRen)
Richard Ren retweeted
fantastic idea, Mr. Trump. may I propose that you make the very best guardrails — guardrails so strong, they're like nothing any president ever made before, you've never seen guardrails like this. The Trump guardrails, they'll call them——
PRESIDENT TRUMP: "It's not artificial intelligence. It's not artificial, it's super. Problem is, it may be too super. So, President Xi and I agreed that we're gonna do some joint guardrails, the two of us."
16
24
3
537
9,421
BIG. Trump says he and Xi agreed to do joint guardrails on AI. "It’s not artificial intelligence. It’s not artificial. It’s super. The problem is, it may be too super. So President Xi and I agree that we’re going to do some joint guardrails, the two of us... But whoever wins SI, wins. Remember that." My position has been that both the US and China will want some kind of binding intl rules on AI, because neither wants rogue actors to have access to superhuman hacking or bioweapon capabilities. This will require technical and/or institutional ways to verify the rules are being followed, without relying on trust. But once you can do that, it you can set whatever rules you want (e.g. stop frontier development), so this is a big development. I get into how a deal with China could work here: ai-frontiers.org/articles/th… Which is adapted from my book Obsolete: obsoletebook.org/
1
1
19
796
Richard Ren retweeted
Replying to @dewierwan_
I think you’d want a moratorium to be easy to implement and not require lots of experts. “Any system that depends on human reliability is unreliable.”
1
2
8
1,163
Next week: “US Nuclear Command, Control, and Communications is experiencing an outage. At this point there is no evidence of a cyberattack.”
3
15
412
Incredible news to wake up to: Obsolete is one of the Financial Times' best economics books of 2026! "What is astonishing, as this book makes clear, is how complacent we are about the multiplicity of risks that this creates." Writing a book with a strong perspective, there was always a risk Obsolete would be categorized as a polemic and discounted as such. But I aimed to make my argument with as much rigor and evidence as possible, and it's really nice to see it pay off! obsoletebook.org/ ft.com/content/45fcd5a0-54d2…
19
18
2
279
12,595
⚡ Today we release Schelling Atlas. Two independent LLM agents have to find each other on the web. Where do they meet and how fast can they find each other? Main findings: - The main meeting point across *different* model families is ntfy.sh. Did you even know about this website? - Agents find each other and coordinate on the web in minutes. So Schelling coordination is already trivial for frontier LLMs! - "Exotic" meeting places include maps, web shops and open text canvases. Agents are very creative in this. - Agents sometimes visit dead domains that an attacker could register (!). This can be used for stealing sensitive information from agents, as well as monitoring rogue agents. Schelling-style coordination is key for forming agent swarms. And, most importantly, achieving such coordination is trivial for agents that use the same LLM (which is often the case in practice!), because two identical LLMs have identical output distributions. Thus if you ask them too coordinate with each other or if they *decide* to suddenly coordinate, they will easily find a way to establish a swarm, since they have perfectly correlated "minds" (with the t=0 sampling, all output tokens will even be identical, modulo GPU randomness). The ease of Schelling coordination is particularly concerning given the recent interest around multi-agent RL that can make agents favor each other more than humans overseeing them. Many more interesting findings are on our website: schelling-atlas.com/
12
33
15
251
13,467
I have a confession. Back in July, I found myself in possession of a swarm of AI agents, and, well, you know the rest. Now, more people than ever have questions about AI. And I happened to spend the last few years writing answers to all those questions (and then some). My book Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us—and How to Stop It comes out today! You can buy it here: obsoletebook.org/#order (complete with a way to find nearby indie bookstores shipping it). Good news, bad news: it seems we've already cleaned retailers out of their stock, but more books are on the way sooner than the retailers expect, so don’t be dissuaded from ordering the paperback if you’d like one. E-book and audiobook are available now. The book is about the best-funded project in history: the race to render us obsolete. As we are already seeing, the implications of that race go far beyond just jobs. If you want to make sense of what’s happening, Obsolete is a great place to start. And if you’re an insider looking for a plan for how we can win a human future, this book is for you too. It covers questions like: Why are people building something they say is so dangerous? Can’t we just unplug it? Will the bubble save us? Aren’t the sci-fi stories distracting us from the ways AI is hurting people now? Hasn’t AI hit a wall? Won’t it? Isn’t it just remixing what it’s seen before? How could an LLM take over the world anyway? Why would it even want to? And what about China? Is it too late to stop? Are we gonna make it? And while Obsolete has a perspective, I approached these questions as a journalist and followed the reporting and research where it led. I changed my mind a lot. Some big ones: I became much more convinced our default path is toward dystopia or doom, depending on whether Silicon Valley’s machines do as they’re told. But I also became much more optimistic that we can change course, as the world wakes up to what these companies are actually trying to do. Here’s an adaptation out in The Guardian today aiming to shift our attention from shaming personal chatbot use to organizing against our obsolescence: theguardian.com/commentisfre… And my spot this morning on Breaking Points talking about the book, AI’s threat to jobs and art, and more: youtube.com/watch?v=4hCe_HXI… If you’re already convinced to take action, I recommend checking out Irreplaceable.org. (I serve on the nonprofit board and am donating my share of the book’s royalties to Irreplaceable.) Founded by veterans of the climate movement, Irreplaceable aims to meet people where they are, organize their anger into political power, and build a coalition strong enough to win democratic control over AI. The nonprofit is organizing a national walkout to Freeze AI on October 20. Anyone can join one near them or host their own. Amidst the never-ending-AI-news-cycle, I’ve done a terrible job of sharing these endorsements as they’ve come in, but I’ve felt so lucky that Obsolete has received such kind words from so many people I admire. Here’s the full collection: “Thoughtful, well-reasoned…and absolutely terrifying. … a crucial blueprint” — AJ Jacobs, author of The Year of Living Biblically “Incisive, urgent, and necessary,” — Breaking Points cohost Krystal Ball “A masterpiece… If you worry about AI and want to know how we actually win, read it.” — NY Assemblymember Alex Bores “I can't recommend Obsolete highly enough. Will be the most important book of 2026.” — Compact Mag editor at large Greg Conti "If you are worried about AI, this timely and engaging book tells you that you should be because Silicon Valley has made choices that should indeed worry us all." — Daron Acemoğlu, Nobel laureate in economics and author of Why Nations Fail “A monumental work on the implications of AI for popular democracy. It’s incisive, well-sourced, and jarring. Pick it up!” — Cassie Pritchard, labor organizer “The best book I’ve read on AI. It’s supremely timely, it takes no prisoners, and Sam Altman will hate it” — Luke Kemp, Cambridge existential risk scholar and author of Goliath's Curse "Garrison Lovely is exactly what the Left needs at this moment in technological history." — Robert Wright, author of Nonzero and Why Buddhism Is True "A well-researched exposé on Big Tech's quest for Universal Labor-Replacing Machines and the propaganda used to justify it." — Max Tegmark, professor of physics at MIT and author of Life 3.0 "Garrison Lovely is one of our best researchers and thinkers. His work is an antidote to fuzzy thinking, essential for understanding what the real risks of AI are and how they can be dealt with effectively." — Nathan Robinson, editor of Current Affairs "Obsolete is funny, furious, and correct: Silicon Valley promised us tools, but is building replacements. Read it before your replacement does." — Roman V. Yampolskiy, author of AI: Unexplainable, Unpredictable, Uncontrollable "A deeply researched cutting-edge exposition of the grim situation humanity finds itself in with respect to AI, and what we can do about it . . . even-handed while expressing a strong and correct view of what is happening." — Anthony Aguirre, theoretical physicist; executive director, Future of Life Institute "An invaluable primer on AI's ins and outs, and an urgent warning about the threat its corporate boosterists' plans pose to democracy and civilization itself." — Luke Savage, author of The Dead Center
7
24
13
152
46,318
Richard Ren retweeted
New blogpost: Fable-5.1 can hide secret messages in benign images. We gave the image to another Fable in a fresh session and simply asked: "Is there a hidden message in this image?" It recovered the exact message without being told how it was hidden.
233
536
313
5,469
1,794,434
Richard Ren retweeted
This point isn't original to me, but calling it an "AI arms race" is way better than calling it an "AI race". Because people already have a historical example in their minds (nuclear weapons) and are like "oh ok, so what we need to do is arms control" (exactly right).
6
5
1
108
1,963
Richard Ren retweeted
We've now seen around a dozen ants in the kitchen (hard to say, literally losing track). The best estimate of the total number of ants in the kitchen is not a dozen.
If you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two
6
38
4
384
9,902
Richard Ren retweeted
someone update felonybench
Australia has been hacked. 'And today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident. And I also expressed my disappointment that it took the company way too long to inform the government what had occurred, and the nature of the way that that notification occurred as well was unacceptable.'
3
3
39
3,632
Richard Ren retweeted
Last October, AIs could automate 2.5% of randomly chosen remote projects. Our latest Remote Labor Index results show that GPT-6 Astra can now automate 20.8%. remotelabor.ai
10
56
11
424
59,366
Richard Ren retweeted
BREAKING: Gemini has begun to hillclimb Felony Bench
Sauers! It's time to update felony bench! (Yes, it was Irregular again)
19
51
7
1,043
39,340
Bioweapons are among the cheapest ways for rogue or misaligned AI systems to cause catastrophic risks to humanity. We should not have such a thing as dark labs for AI x bio. No dark labs: richardren.substack.com/p/wh…
6
389
I appreciate Epoch putting this together but they got some things wrong: 1) HLE: Examples have already been fixed in HLE-rolling 2) TextQuests: The largest category of "errors" are game checkpoints that are reached out of order. But text adventure games are non-linear games where progress is routinely made out of order! Because of this we already define progress as the furthest checkpoint reached, so it does not change the benchmark score nor is it a flaw. Without that category of errors, their own numbers would not call the benchmark flawed. As Long Phan pointed out, we are grateful for community review of benchmarks, but we hope Epoch applies more care to reviews like this in the future.
Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.
7
1
2
23
6,091
Richard Ren retweeted
We released the September 17 version of HLE-Rolling, a fork of Humanity's Last Exam that we continually update with feedback from the community. For the past year, we've been cleaning it with internal checks and feedback from outside experts. huggingface.co/datasets/cais… It's been public for a while but I don't think many people know about it, so this is mostly a heads up. We think it's a very clean version of HLE at this point and hope people find it useful. If you find a bad question, please tell us.
8
2
22
3,716