Host of Doom Debates — disagreements that must be resolved before the world ends.
San Jose, CA
Joined July 2008
- Tweets35.7K
- Following1.5K
- Followers41.1K
- Likes149K
Pinned Tweet
NEW: “AI Chernobyl” Will Scare Trump and Xi To Order An Immediate Pause
The famously charismatic AI policy duo of @JeremieCHarris and @Harris_Edouard, physicists and cofounders of Gladstone AI, joined me on Doom Debates to discuss their Trump–Xi handshake deal scenario and the exact degree to which Yudkowsky is right.
Ed & Jeremie expect we'll see an AI warning shot that finally scares Trump and Xi to negotiate an AI slowdown. In the meantime, the US better get various enforcement tools ready to make a future deal stick.
Their background is impressive:
👉 They pivoted their careers around the AI threat in 2016, long before ChatGPT brought AI into the mainstream
👉 They’ve built relationships with senior government decisionmakers without a prior national security background
👉 They have an incredible talent for communicating about this with the average Joe (Rogan)
Topics covered:
⬜ The GPT-3 wake-up call
⬜ AI risk outside the EA community
⬜ The Harris brothers' backgrounds
⬜ Yudkowsky and quantum many-worlds
⬜ SharpestMinds and leaving academia
⬜ Briefing the US Government
⬜ How Gladstone AI pays the bills
⬜ The Trump-Xi phone call
⬜ AI Chernobyl
⬜ Verifying China's AI slowdown
⬜ Espionage and lab security
⬜ Offensive operations and deterrence
⬜ The US-China AI lead
⬜ Why a treaty won't happen
⬜ Warning shots and extinction
⬜ What's Your P(Doom)™?
⬜ PauseAI and movement politics
It's a uniquely fascinating episode about the geopolitics of saving humanity from ASI. Highly recommended! 👇
I dunno if it's related to International Hack Everything Day but Google Firebase Realtime Database in us-central-1 has been degraded all day: status.firebase.google.com/i…
Liron Shapira retweeted
NEW: “AI Chernobyl” Will Scare Trump and Xi To Order An Immediate Pause
The famously charismatic AI policy duo of @JeremieCHarris and @Harris_Edouard, physicists and cofounders of Gladstone AI, joined me on Doom Debates to discuss their Trump–Xi handshake deal scenario and the exact degree to which Yudkowsky is right.
Ed & Jeremie expect we'll see an AI warning shot that finally scares Trump and Xi to negotiate an AI slowdown. In the meantime, the US better get various enforcement tools ready to make a future deal stick.
Their background is impressive:
👉 They pivoted their careers around the AI threat in 2016, long before ChatGPT brought AI into the mainstream
👉 They’ve built relationships with senior government decisionmakers without a prior national security background
👉 They have an incredible talent for communicating about this with the average Joe (Rogan)
Topics covered:
⬜ The GPT-3 wake-up call
⬜ AI risk outside the EA community
⬜ The Harris brothers' backgrounds
⬜ Yudkowsky and quantum many-worlds
⬜ SharpestMinds and leaving academia
⬜ Briefing the US Government
⬜ How Gladstone AI pays the bills
⬜ The Trump-Xi phone call
⬜ AI Chernobyl
⬜ Verifying China's AI slowdown
⬜ Espionage and lab security
⬜ Offensive operations and deterrence
⬜ The US-China AI lead
⬜ Why a treaty won't happen
⬜ Warning shots and extinction
⬜ What's Your P(Doom)™?
⬜ PauseAI and movement politics
It's a uniquely fascinating episode about the geopolitics of saving humanity from ASI. Highly recommended! 👇
“You can just do things”: pre-AGI grindset
“You can just be meatpuppeted by an agentic loop”: centaur-era god mode
I am once again asking for an institution that certifies whether minimum standards of discourse are being met
Ladies and gentleman, @bhorowitz's latest argument:
💁♂️ Bill Gates “induced” COVID via his pandemic preparation efforts. Similarly, Anthropic "manifests" human extinction from ASI because the very act of worrying drives them to create AI that thinks it's smarter than us.
Liron Shapira retweeted
Ladies and gentleman, @bhorowitz's latest argument:
💁♂️ Bill Gates “induced” COVID via his pandemic preparation efforts. Similarly, Anthropic "manifests" human extinction from ASI because the very act of worrying drives them to create AI that thinks it's smarter than us.
Ladies and gentleman, @bhorowitz's latest argument:
💁♂️ Bill Gates “induced” COVID via his pandemic preparation efforts. Similarly, Anthropic "manifests" human extinction from ASI because the very act of worrying drives them to create AI that thinks it's smarter than us.
I think he may have actually managed to top “The key to nuclear safety is nuclear proliferation”: xitter.cf/liron/status/184966065…
Ben Horowitz says nuclear proliferation is GOOD?
I disagree. I thought we all agreed nuclear proliferation is VERY BAD.
If @bhorowitz and @a16z are saying this about nuclear weapons proliferation, I would take their AI policy arguments with a grain of salt.
I also see @paulg as epistemically compromised:
Liron Shapira retweeted
I checked this and my version, on the highest available depth, gave the same answer. chatgpt.com/s/cx_6ad201b0822… I am cancelling my subscription. @sama @OpenAI @gdb
Liron Shapira retweeted
I think it's time to update that I was overrating the complexity class NP.
Finding efficiently-verifiable solutions indicates high intelligence, but real-world optimization has more non-efficient-verifiability to it than I expected.
What complexity class captures this — PSPACE?
Adding my 2 cents at a quick glance:
1. This result has no implications about intellidynamics (agent theory) as neural activations are separated from the dynamics of our disempowerment by multiple levels of abstraction
2. This is the kind of verifiable result that AI itself will pick off early in the “just get the RSI’ing AI to do the alignment research” regime and serves as a litmus test for people who think that paradigm warrants optimism.
Me: Mechanistic interpretability sounds like a good idea, yes. If we could learn something about how an AI was thinking, in enough detail to predict in advance what it would think even as the thought departed from human, that would be a bare beginning on the sort of understanding you'd needed to get started on AI alignment that wasn't just about hammering on an AI from the outside with SFT and RL.
AI guys: We found a neuron that lights up when the AI thinks of the Eiffel Tower!
Me: Cool. That does sound like it'd be a prerequisite to getting started on the important part eventually, someday.
AI guys: Oh NO! There are MULTIPLE neurons that light up when the AI thinks about the Eiffel Tower! And some of those neurons also participate in OTHER concepts! We're calling this unexpected development 'polysemanticity' and 'superposition'.
Me: That sure sounds like a thing that would happen, yes. I cannot say that I am impressed by you acting surprised about it. Let me know when you've gotten far enough in mechanistic interpretability to get started on the important part.
AI guys: We have found that with BIG AIs you can often point to a SINGLE neuron that lights up when the AI thinks of the Eiffel Tower! Understanding AI thought processes is gonna be EASY PEASY!
Me: Cool. Let me know when you've advanced to the point of getting started.
Impressive and totally shuts down people (e.g. Sam Altman) claiming the present scenario is outside of Yud’s prophesied range of trajectories
Huh. I didn't realize @ESYudkowsky in 2008 explicitly ruled in 'we might build AGI without any deep new insights'. Seems like we're currently moving from scenario 2 (programmer-driven) to scenario 1 (AI advances AI).
Also, apparently 2013-Eliezer's median takeoff year was ~2035!
