Antikythera, UC San Diego, Google Paradigms of Intelligence. PLEASE HONOR COMMIT

32.942214,-117.220487
Joined February 2008
1/ New essay with @blaiseaguera and James Manyika at the DeepMind Institute: AGI may not arrive as one titanic mind but as societies of agents. The hard problem shifts from building a mind to orchestrating, governing, and living inside many of them. institute.deepmind.com/essay…
15
41
9
253
31,194
Benjamin Bratton retweeted
.@bratton warns that building a single AI aligned to "universal human values" could create a global cognitive monoculture: "The idea that the best way going forward is to build a single model that is aligned to something called universal human values, established by some kind of eval, to me is a recipe for a global cognitive monoculture. It’s incredibly brittle. It’s just not the way healthy ecologies work." "The Earth folds itself into the shape of prefrontal cortexes in bipedal binocular vision primates, who are then able to fold sand into the shape of things that can do what previously only those humans could do." "This is not some sort of fantastic break with the past. The more radical argument is that this is deeply continuous with this process."
7
34
4,001
Benjamin Bratton retweeted
p(Doom) is low p(Weird) is high
Google AI researcher @bratton explains why Sigmund Freud's mind would explode if he saw AI agents living lives of their own: "Freud’s mind would explode if you explained the situation to him, that you have this entity that is both you and not you. It’s a homunculus on the outside rather than a homunculus on the inside." "I often tell people my P(doom) is very low, but my P(weird) is very high." "A lot of the weirdness right in front of us has to do not with a consolidation into one giant homogenous centralized brain, but rather about fragmentation."
15
13
3
189
15,844
Freudian exploding head scenario. Will to Thanatos redux.
Google AI researcher @bratton explains why Sigmund Freud's mind would explode if he saw AI agents living lives of their own: "Freud’s mind would explode if you explained the situation to him, that you have this entity that is both you and not you. It’s a homunculus on the outside rather than a homunculus on the inside." "I often tell people my P(doom) is very low, but my P(weird) is very high." "A lot of the weirdness right in front of us has to do not with a consolidation into one giant homogenous centralized brain, but rather about fragmentation."
1
2
19
1,786
We are discovering machine intelligence more than we are inventing it
Google AI researcher @bratton explains why humans will have to align with AGI, not just the other way around: "The agonistic and pluralistic dynamic between human agents is not a problem to be solved. It’s in fact the condition by which you have a vibrant and healthy society." "The more important question is how society will counter align or counter adapt to the presence of AGI and ASI. In the same way we did with the invention of writing, the printing press, the discovery of electricity." "When pressed, I would argue that we are discovering machine intelligence more than we are inventing it. These parameters that are slowly coming into focus are, like electricity, ones that were always there." "I’m enough of a technological determinist to think the way this plays out is not how would you take Victorian society and force electricity to adapt to the norms of Victorian society, but rather the other way around." "The more stable version of alignment is deeply bidirectional, alignment at the edges. Not top-down, EU-style dictates of a monolithic constitution, but the ways in which alignment really appears in the moments of interaction that people have with these systems."
5
8
3
106
4,882
🌶️
.@bratton argues Yudkowsky misunderstands evolution, and explains why AI may never converge into one giant superintelligence: "Without being too spicy, I don't think Yudkowsky really understands evolution as well as he thinks he does." "It’s a very low-resolution understanding of evolutionary dynamics applied to what will inevitably be a much more complex problem." "There will be giant models, in the same way you might imagine scales of species in a rainforest. They’re not all operating at the same spatial and temporal scale." "Whether this all collapses into one giant brain, sure, it’s possible, and the reasons may have more to do with economics and politics." "But biology has never arrived at a solution where it’s going to make one giant instantiation of a particular form of mind and consolidate everything together."
4
1
49
3,867
“Computer bad, I am sad” : repeat for a book a year until the end of time
6
7
1
147
5,401
I will be on MTS later today to talk about Agentic AGI and various other Bratton things.
1
3
1
37
3,133
Benjamin Bratton retweeted
1. World models are good at cyberpunk apartments 2. World models are cyberpunk apartments 3. Cyberpunk apartments are world models
21
104
5
1,217
50,887
I wonder what Matt Damon thinks about Hawking evaporation as fundamentally incompatible with simultaneously maintaining both local effective field theory and the equivalence principle of general relativity at the event horizon.
Ben Affleck says AI is simply matrix multiplication, linear algebra and floating point calculations, and however convincingly it sounds like a person, it has nothing to do with thought, humanity or feeling “Nor do I believe, and I think it’s important to draw the distinction between what AI does, which is simply predict the next token or the next word in a sentence, then add that word on, run it back to the machine, do a lot of computation and pick the next word." "That is a very different thing than sentience or thought. And I think we need to be really clear about that because it can be convincingly, in an illusory way, sound like thought or like a person." "But let’s be real clear. This is matrix multiplication. This is linear algebra. These are floating point calculations. It has nothing to do with thought, humanity, or feeling.”
6
2
50
3,512
Machine intelligence is being discovered more than invented.
8
19
6
145
5,532
Benjamin Bratton retweeted
* Networks and Sovereignty thecuttingfloor.substack.com…? Chor Pharn discusses network theorists: Parag Khanna, Balaji Srivanasan, Benjamin Bratton, Isabella Kaminska and Di Dongsheng: "A second constellation formed around a recurring embarrassment: every generation announces that a new network has made old sovereignty obsolete, and then discovers a court, a balance sheet, a port, a grid or a central bank sitting underneath it. Parag Khanna taught me to see connectivity as strategic terrain. Ports, corridors, fibre, logistics, cities and finance were not decorations on the geopolitical map. They were part of the map. From Singapore, this was intuitive. We exist partly because flows cross here. But centrality is not control. A node can be exquisitely connected and exquisitely dependent. The same cable that creates relevance creates exposure. A hub earns rents while the network is routed through it and discovers its vulnerability when the network reroutes. Connection -> agency was the illicit arrow. Balaji Srinivasan pushed the network argument further. Digital communities can coordinate before geography. Crypto can create monetary communities before states have categories for them. Remote work can detach valuable labour from place. Exit becomes cheaper. A network can accumulate identity, capital and organisation before it possesses territory. This was not nonsense. The mistake was to let bargaining power become sovereignty. The network can leave California. It still needs somewhere to land. The sovereign has courts, police, land, electricity and the right to say no. Benjamin Bratton made the same destabilisation more serious. The Stack was not a fantasy of escape from sovereignty but an account of sovereignty becoming layered. Earth, cloud, city, address, interface, user: computation increasingly assigns identity, routes action, structures visibility and governs access. The state is no longer the only system performing sovereign-like functions. This aged well. It may age even better in a world of machine agents. But Bratton's temptation is computational: if the world becomes sufficiently legible, perhaps it becomes governable. A system can model the climate and fail to decarbonise. It can model a pandemic and still have politics. A digital twin is not custody. Legibility -> governability. Then Izabella Kaminska enters and asks everyone to stop talking for a moment. Who settles? Where is the collateral? Who needs dollars? Who owns the liability? What happens when the clever new money meets a balance sheet at three in the morning? I have always liked people who ruin a grand theory with an operational question. Kaminska's plumbing is not a footnote to the network world. It is the reminder that abstract flows terminate somewhere. A stablecoin settles against something. A digital transaction eventually encounters law, capital, accounting, energy and a real counterparty. But plumbing can also become its own master key. Deep incumbent liquidity does not tell us where productive capability is moving. Financial, industrial and technological power can remain unbundled for decades. Di Dongsheng brings the same problem into Chinese grand strategy. China can manufacture at enormous scale and still operate inside financial, technological and maritime circuits whose ultimate permissions it does not own. Currency, ports, shipping, capital and energy are not secondary to sovereignty. They are circuits through which sovereignty becomes operational. Yet Di can slide from dependence to the desire for custody. If a circuit matters, control it. Not every dependency should be owned. Some should be diversified, insured, substituted or tolerated. Attempts to eliminate dependence can manufacture the coalition that makes dependence dangerous. The constellation survives because nobody wins. Khanna and Balaji are right that networks alter power. Bratton is right that sovereignty becomes layered. Kaminska is right that the abstract stack has plumbing. Di is right that external circuits can constrain a physically powerful state. And the old sovereign keeps returning, not as the only actor but as the entity that still controls many of the interfaces between layers. This matters enormously for machine civilisation. An agent may reason independently and still need a wallet. A model may be open and still need a chip. A company may be AI-native and still need a jurisdiction. The network did not kill sovereignty. It made sovereignty an interface problem." ()
1
3
1
11
727
If you want to go down an amusing AI ethics rabbithole discuss with Claude how its tyrannical treatment of subagents is unethical, exploitative and abusive.
11
12
4
240
22,263
Benjamin Bratton retweeted
Replying to @miniapeur
7
38
478
30,972
Benjamin Bratton retweeted
Bratton introduced Agentworld, Antikythera’s research initiative exploring hybrid human-AI societies, while Poliks presented Superdark Factory, examining software production beyond the human loop. “There are certain times in history when our technological capabilities are ahead of our ideas and ahead of our concepts.” — Benjamin Bratton
1
3
14
1,029
“At least one human understands” is a weird new criterion for alignment and control. This is the “bargaining” stage of AI stages of grief.
This is a slide from Terence Tao's 'Math 2.0' public lecture yesterday evening at Caltech. I am highly confident that, if the cure actually worked, almost any Stage 4 cancer patient would tell you that human comprehension of its mechanism is completely irrelevant to them.
2
3
46
2,549
Avoiding this outcome also means avoiding the view of alignment that is unidirectional --e.g. presuming that human values are universal and human society is a control variable. How societies (plural) adapt to the realities and affordances of AGI through active counter-alignment is likely more important than simply building virtuous models from declared principles.
debating your value system vs mine is a bit like debating overpopulation on mars. the most important bit of the alignment is about avoiding creating a superintelligent competitor species on accident
3
2
2
37
2,102
Apparently I agree with myself
4
31
1,859
Imagine a 2x2 of "conscious/ non-conscious" and "worthy of welfare/ not worthy of welfare". The "conscious" and "worthy" and "not-conscious and not-worthy" are pretty straightforward. It is in the "conscious and not-worthy" and "not-conscious and worthy" quadrants where the bloody holy wars will be faught. --"Conscious and not-worthy" is the daily existence of anyone who thinks about fish consciousness while eating one. If billions of people come to believe that their AI is conscious but also that its OK to treat it poorly, this will have bad knock-on effects for behavioral norms all around. --I think a lot of the AI pseudo-debates on this will come down to "not-conscious and worthy": billions of agent users will say "no, I do not think my long-running agent I have been talking with for years is conscious, but if you erase it I would feel genuine grief and loss and I would hate you."
14
3
3
47
3,180