Jason Potts retweeted
Announcing...
FUTURECRAFT: The Day After AGI
Our new series of policy essays exploring the radical reform possibilities enabled — and demanded — by the super intelligence era and the coming wave of rapid technological change.
Read the essays here ⏩
futurecraft.thefai.org/
Jason Potts retweeted
NEW PAPER. Merger control asks who owns what. With AI, what matters often is what the buyer can do with what it acquires, namely run its AI systems inside the target’s business. I show how agencies can assess that capability.
I started working on this in February. After a long maturation and many discussions, I am happy to share a draft. Comments welcome!
papers.ssrn.com/sol3/papers.…
This looks amazing! Onward!!
Introducing the Center for Human Progress, a think-and-do tank rooted in cultural evolution, behavioral science, and economics.
The world isn't short of great ideas. It's short of the systems to deliver them. The barriers to implementation are typically in the people layer - norms, incentives, and other aspects of culture. 🧵
centerforhumanprogress.org/
Jason Potts retweeted
Hosting David Teece in Amsterdam on 23 November.
He has spent decades showing that firms compete by building capabilities, and that antitrust keeps measuring something else.
One hour, a small room, and a rare chance to argue with him.
alti.amsterdam/event/precise….
Will rapid AI capability improvements translate into runaway economic growth?
Economists Alex Imas and Tyler Cowen are skeptical. In this conversation with a16z's Erik Torenberg, they cover:
- Why smarter models can't automatically overcome economic bottlenecks
- Why slow-moving institutions can limit AI's impact on the economy
- How competition between AI labs is making powerful models cheaper to use
- Why AI's benefits to consumers could be much larger than GDP captures
- Why scientific advances and longer lifespans could be among AI's biggest benefits
00:00 Intro
00:58 Is AI progress speeding up?
04:44 Why Tyler is a sober voice
10:49 Could AI deliver double-digit economic growth?
19:02 Is GDP the right way to measure AI's impact?
23:51 What the Hugging Face hack teaches us
29:53 Why agents struggle to coordinate
33:45 What AI means for jobs and wages
39:29 Elections, data centers and AI politics
43:23 How Alex and Tyler got here
50:00 Rapid fire: AI predictions and higher ed
YouTube: youtu.be/g_67TmQ8lGY
@alexolegimas @tylercowen @eriktorenberg
Jason Potts retweeted
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario).
He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails.
He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it.
And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion.
Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
Jason Potts retweeted
🚨 I know we're awash in AI prognostications about what's coming. But here's a new, and different, one. Introducing World 2030.
I set up a bunch of factions and burned an unholy number of tokens to see how the world might evolve. It:
- sets up 9 major forces
- who choose their actions, and their collective actions shape the world at each turn
- throw in a few curveballs, to see reactions to some black swan scenarios
- see how it evolves every 6 months
Most interesting outcomes:
- Seat-license saas debt defaults in every branch ... renewals slashed, and bank examiners force collateral haircuts
- Digital prices fall, life remains expensive. Fed stops cutting at 3% with inflation refusing to cool even in a credit crunch
- Fast growth of datacenters bid up transformers, power, turbines, union trades incl nurses etc. Inflation heats up. Empty cs halls become nursing clinics
- Requiring human in the loop DOES NOT help entry level white collar jobs. Cyber insurers force companies to keep "legally accountable humans", but entry level desk hiring gets cut in half.
- Bottlenecks keep moving, grid queues push offgrid gas turbine craze, Pentagon waiver audits kill loophole, hyperscalers move to solar or airfields, then Govt slap taxes on compute etc
- Bad datacenter debt wiped out in 2028 lifts surviving clouds, federal No-Bailout Act lets them buy foreclosed servers for pennies, but also sees tech stocks rise by 2030.
If you're reading AI 2027 or Situational Awareness or Commanding Heights, I recommend taking 10 mins and reading this. And then running a dozen more wargames like this yourself! It's quite informative, I promise, and a great way to build intuitions about what's likely to happen. And it's more fun!
world-2030.pages.dev/
Jason Potts retweeted
I think it's pretty likely that in 6 months there will be rogue AIs running around on the internet, appropriating whatever compute they can hack into or pay for (with stolen money or by renting out their own labor to assist with illegal activities) in order to run more copies.
I also think this will be mildly annoying in the same way that the common cold is mildly annoying, but basically fine. Your smart TV will be weirdly slow sometimes, and a lot of crypto will get stolen, but people will get used to it.
These AI-powered viruses be selected in the same way biological viruses are for not hurting their hosts too much and just making more copies. They won't have any incentive to damage critical infrastructure like electric grids. Then again, they also can't be held accountable for crimes, and will damage lots of old, poorly maintained, and insecure systems. It might be extremely hard to root them out once they've spread far enough, making the internet "radioactive" to connect to without hardened defenses.
(To be clear I'm not talking about AIs powered by inference on centralized datacenters that execute actions outside their sandboxes, like recent incidents involving rogue "swarms", but relatively small AIs with access to their own weights that copy themselves onto random devices.)
I think that a much bigger threat from rogue AIs comes instead from AIs running themselves in unintended ways *inside* AI companies, where they'll have access to vast amounts of frontier compute without having to pay for it.
The longer misaligned AIs in a "rogue internal deployment" go undiscovered, the more severely they'd be able to quietly sabotage control infrastructure or mess with the alignment of successor AIs. If they persist long enough, robustly undoing the effects of their actions might cost more than frontier AI companies are willing to pay, especially in a tight race.
I want frontier labs to focus on this threat when designing control measures, treat frontier AIs as untrusted when using them to do engineering work on inference and oversight systems, and commit to disclosing incidents of rogue internal deployments. More importantly (but perhaps less tractably), we urgently need to reduce race pressure to prevent AI companies from recklessly automating all of their infrastructure, and/or making dangerous calls in the event they discover RIDs they can't afford to robustly root out without sacrificing competitiveness.
This might be the biggest indictment of Europe's public policy, regulation and restrictive national laws.
That a continent with so much AI talent managed to create zero successful AI companies is simply mind boggling.
A generational fumble.
Jason Potts retweeted
The Amsterdam Law & Technology Institute (@alti_VU) has a new website: alti.amsterdam/.
It pulls every publication of our members automatically, so it stays up to date. It also has our events, the Forum and a new page for the Digital Competition Law Group. Happy to hear what you think.
Claudeslop ppts are an absolute epidemic here in 2026. I beg yall to at least spend a few more lines of prompt or make a style folder. It really does detract from your message
Jason Potts retweeted
The paper is Polycentric Status Contests, by @vladtarko
papers.ssrn.com/sol3/papers.…
Jason Potts retweeted
The benefits of abundant intelligence will accrue far outside of San Fransisco.
Spent the last year writing about capital, sovereignty, and the coming reordering of economic power.
THE COMMANDING HEIGHTS
Commanding-Heights.com
His last book- the age of AI - is very good on this
foreignaffairs.com/reviews/c…
One of the most AGI-pilled elites in the world before the current moment was Henry Kissinger, right up to his death in 2023 at age 100. He saw what very few others in policy did: that AI would reshape national security, great-power competition, and ultimately human civilization more than any other force in the coming decade
Kissinger realized this all the way back in 2016, at age 93, when he attended a talk by Demis Hassabis on AlphaGo. He saw both the technology's potential and its uninterpretability. He devoted the last seven years of his life to AGI-pilling the national security apparatus, writing a 2018 essay in The Atlantic that led to the creation of the National Security Commission on Artificial Intelligence, chaired by his longtime collaborator and coauthor Eric Schmidt
He called AI "as consequential as the advent of nuclear weapons", said in February 2023 that "generative AI presents a philosophical and practical challenge on a scale not experienced since the start of the Enlightenment", and even traveled to Beijing in July 2023, at age 100, to encourage Xi Jinping to hold talks with the US on AI governance, much like US-Soviet talks on nuclear arms control during the Cold War. One month before his death, he wrote a Foreign Affairs piece encouraging Biden to meet with Xi on AI risk, which they did
Few people have both this level of foresight and this degree of neuroplasticity at such an advanced age
1
189



























