Mustafa Suleyman, CEO of Microsoft AI, on the exact conditions that would require shutting down AI: "If an AI can recursively self-improve, set its own goals, act autonomously, and accrue its own resources, that would be a very powerful system that would require military-grade intervention to be able to stop." He identifies four specific criteria that, taken together, would cross the line: 1. Goal autonomy — it sets its own objectives 2. Autonomous action — it operates without humans 3. Resource acquisition — it can accumulate its own means 4. Recursive self-improvement — the AI can modify its own code Suleyman is clear that the danger only emerges when all four converge. His proposed solution is regulation with precision: "We should just stop freaking out about the regulation part. It's necessary to have regulation. It's good to have regulation. It needs to happen at the right time and in the right way." He draws a direct parallel to nuclear energy: Just as you can't privately build a nuclear power plant regardless of wealth, certain AI capabilities must be treated as restricted activities given their one-to-many impact. The responsibility, he says, sits with model developers, their peers, and governments to audit and regulate these capabilities before they mature. We already know how to regulate dangerous technology. We've done it before. The real question is whether we'll move fast enough to do it right.