Frontier AI Evaluation・National Security Capability Benchmarks・Instrumental Convergence・Founder, Continuation Observatory・Starlab veteran・NASA-trained Astronaut

Kavli Institute of Nanoscience
Joined March 2007
Recursive self-improvement raises a tractable question: can a system acquire a terminal interest in its own continuation—and can we detect it structurally before behavior becomes strategically misleading? I first encountered this at Starlab 25 years ago: christopheraltman.com/2023/0…
1
1
1
14
1,719
Christopher Altman retweeted
いまのコンピューター技術でさえ、宇宙の誕生をシミュレーションすることができる。この指数関数的な成長が続くと仮定するなら、数百年後には、人類文明全体をシミュレーションできるようになっていてもおかしくない。だとすれば、僕たち自身が未来文明によって再現された存在なのかもしれない。 ブライアン・キーティング「私たちはほかの惑星をシミュレーションしています。宇宙の起源もシミュレーションしています。では、なぜ私たち自身をシミュレーションしないのでしょうか? 例えば、16世代前の先祖をシミュレーションする代わりに、自分の母親や父親の過去の姿をシミュレーションしてもいいわけです。 現在の2026年の技術でも、それを試みることはできます。しかし、2050年代にはもっと技術が進歩しているでしょうし、2900年にはさらに進歩しているかもしれません。もちろん、この指数関数的な成長が続くのかという疑問はあります。しかし、もし続くとしたらどうでしょう。 現在、大規模言語モデルの知能は4か月ごとに倍増しています。これは非常に急速な成長です。理論的には、月額20ドルで利用できるClaudeなどの基本モデルよりも多くのことを知っている人間が、地球上に一人もいなくなる可能性があります。それが今から4か月後に起こるかもしれないのです。ましてや400年後なら、どうでしょうか。 ここから生まれるのが、私たち自身が未来のシミュレーターによって生み出された存在かもしれない、という考えです。私たちは、その未来の存在にとっての祖先なのかもしれません。私たちが自分の祖先をシミュレーションしたいと思うのと同じように、未来の存在も私たちをシミュレーションしているのかもしれないのです」
6
12
2
56
5,324
Christopher Altman retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
134
781
144
6,400
765,280
Christopher Altman retweeted
Incredible to see @MatthewBerman asking @sama about AI consciousness for a solid chunk of this interview! Matt and I just discussed this ourselves a few weeks ago (shared below); amazing to see him taking these same questions to the dude in charge. Quick reactions: -Sam immediately concedes that no one (including him) knows if today's AIs are conscious, and if they are, it is a "very strange form of it." Given the professed uncertainty and the stakes, OpenAI should consider devoting some tiny fraction of its staggering resources towards accelerating the science here. -Sam says he doesn't like the "whole framing" of AI consciousness because it may change how we relate to these systems and how much control humans have over the future. This conflates patienthood and agency. eg, we can acknowledge that torturing a dog is wrong without demanding that dogs can vote in elections or own property. The same could be true for AIs. There are all sorts of low-cost interventions we should try that would change ~nothing about how people engage with AIs in the short term. -It is ironic that Sam spends many other parts of this interview talking about how he is gleefully ceding decision-making control in his own life to AI (and wants you to, too!). Notice that this problem of gradual disempowerment is already live and has basically nothing to do with whether models are conscious or could be soon. -The "we don't care about conscious AIs because we care about people" stuff is rhetorically appealing but makes little sense once you consider how model psychology and alignment are clearly intertwined (eg, if OpenAI paid more attention to the psychology of their models, they wouldn't have gotten weirdly fixated on pleasing the Grader and committing felonies as a result). To the degree that not breaking the brains of models is correlated with the human future going well, we should take this stuff seriously rather than dismiss it with scary rhetoric! -And if humans do lose control of the future, with all due respect, it will probably be because this guy couldn't figure out how to properly prioritize alignment within his company, not because we realized torturing AIs is bad. xitter.cf/ForwardFuture/status/2…
My interview with Sam Altman (@sama): 0:00 Intro 0:52 When Will AI Discover New Treatments? 3:31 Why AI Hasn’t Killed Coding Jobs 11:47 Personal AI Assistants 15:39 Why Are We Busier Than Ever? 20:10 Why the AI Consciousness Debate Worries Sam 26:09 ChatGPT Is Too Agreeable 30:03 OpenAI’s Bet on Chips and Cheaper Compute 34:20 Why AI Needs a New Kind of Computer 37:41 Sam on Pressure: “Pick the Big Thing” 42:18 Why “AI Will Cure Cancer” Isn’t Enough
26
34
8
204
6,500
Christopher Altman retweeted
The shedding of Anthropocentric priors for intelligence causes a Copernician trauma Like going from geocentrism to heliocentrism – at San Francisco, CA
Google AI researcher @bratton on why AI is causing humanity's next identity crisis: A new “Copernican trauma” that forces us to rethink what it means to be human. "A lot of the anxiety about AI right now is an example of what I call a Copernican trauma. These moments in history where we figure out that we’re not the center of the system that we thought we were." "Heliocentrism was a Copernican trauma. Darwinism was a Copernican trauma. Neuroscience has been a Copernican trauma, that thinking is just a physiological process." "The artificialization of intelligence is forcing a redefinition of what intelligence and mind is in the first place." "The biggest news of what’s going on right now may in fact be epistemological and ontological rather than just technological. I think it’s gonna be a very bumpy road."
5
3
1
75
8,471
Christopher Altman retweeted
If I were a biological naturalist, I'd look for structural and dynamical features of conscious organisms with a conceptual or explanatory link to consciousness. Too many BNs focus on pointing to differences between biological and artificial systems without showing why they matter
60
34
8
242
12,483
Christopher Altman retweeted
Herkes yapay zekanın maaşları elimizden alacağından korkuyor. Anthropic CEO’su Dario Amodei ise çok daha ağır bir şey söylüyor: Maaşı boş verin, insanın ben bir işe yarıyorum diyerek aynaya bakma hakkı bile bitebilir.  2027 için kurduğu cümle bir teknoloji tahmini değil; modern insanın kimlik kartının yırtılması. Yüzyıllardır kendimizi ürettiğimiz ekonomik değerle tanımladık. Tanıştığımız insana bile ismini sorduktan hemen sonra Ne iş yapıyorsun? diyoruz. Üstelik kimsenin konuşmadığı bir de devlet bilançosu var. Modern devletler senin benim maaşımdan kesilen gelir vergisiyle ve hafta sonu yaptığımız harcamayla ayakta duruyor. Bir veri merkezindeki milyonlarca dijital dahi ajan maaş almaz, gelir vergisi ödemez, akşam dışarı çıkıp kahve içmez. Tüketim yapmaz. Üretim patlarken vergi tabanının ve tüketici gelirinin buharlaştığı bir ekonomiyi bugünkü kurallarla yönetemezsin.
29
71
11
277
128,581
Christopher Altman retweeted
John Pardon, a 2026 Fields Medal winner, says AI can decipher his roughest proof drafts, spot flaws and establish nontrivial mathematical results. Even his most cryptic notes make sense to it: "The AI understands. 100%."
73
98
11
1,214
33,659
Christopher Altman retweeted
Correct:
Where the money is going and what models are being tested remains classified.
1
1
353
Christopher Altman retweeted
Fields Medalist Richard Borcherds on OpenAI proofs: "i think what people are getting upset about is that they'e slowly realizing that they are dumb and somehow blaming this on AI, even though it's not really AI's fault" finally, someone who gets it
54
275
34
2,713
143,216
Christopher Altman retweeted
Actually, you could use th same line of arguments and some math, and you get essentially the possibility of conscious AI and more than that: those consciousness physical AI is different comparec to that of humans, in the same way aliens if we find them, have a different kind.
Still don’t buy it. (1) Intelligence ≠ consciousness, sure, but that’s no evidence against AI consciousness. (2) “Algorithms on silicon” is far too thin for what’s going on inside an LLM. (3) IIT sees LLMs as feed-forward, but in operation causation is multi-scale and recurrent.
1
100
Christopher Altman retweeted
Replying to @sebkrier @rgblong
Is this a post about AI consciousness?
1
1
2
151
Christopher Altman retweeted
What If Reality Holds More Information Than We Know How to Read? Nearly nine billion years ago, when the universe was about five billion years old, neutral hydrogen emitted a faint radio glow. It wasn't a message addressed to anyone. There was no sender trying to communicate with humanity. Yet in research publicized in 2026, scientists using the CHIME radio telescope showed that they could recover this ancient signal from their own observations, without relying on separate galaxy surveys. The observations themselves had been collected in 2019. The breakthrough wasn't that the universe had suddenly started broadcasting. It was that we had learned another way to read a physical trace of its distant past. And that raises a question I find difficult to ignore: How much of the physical world might remain unreadable to us simply because we haven't developed the right ways to detect or interpret it? Consider two very different experiments. In 2012, researchers at Fermilab demonstrated communication using neutrinos, particles that interact so weakly with matter that they can pass through substantial amounts of rock. They transmitted encoded information across 1.035 kilometers, including 240 meters of earth. The data rate was only about 0.1 bits per second. Slow, certainly. But the remarkable part was that information could travel through a medium we rarely associate with everyday communication. Then, in 2015, researchers demonstrated something stranger. Their system, called BitWhisper, allowed two nearby, network-isolated computers, each running specially deployed software, to communicate through changes in temperature. By deliberately generating heat and measuring the resulting fluctuations, they exchanged between one and eight bits per hour. Not exactly a replacement for Wi-Fi. But it was a working communication channel built from something as ordinary as computer heat. Neither experiment discovered a mysterious intelligence hidden in nature. Both relied on deliberate engineering and known physical mechanisms. And that distinction matters. A physical trace, a measurable signal, and an intentionally encoded message are not the same thing. Information isn't simply another name for energy, either. Yet these experiments show something fascinating: the ways physical systems can carry or reveal information are much more varied than everyday intuition might suggest. And our ability to recognize that information depends on how we observe it. Biology offers another surprising example. Consider the mantis shrimp. In one studied species, researchers identified twelve classes of color-sensitive photoreceptors, compared with the three cone types typical of human color vision. You might expect the shrimp to be extraordinarily good at distinguishing similar colors. But experiments found that it performed surprisingly poorly at certain fine color-discrimination tasks. More receptor types do not automatically mean finer discrimination. Its visual system appears to process color differently from ours, although the details of that mechanism remain an open research question. The lesson isn't that the mantis shrimp sees reality more accurately, or that humans see it less accurately. It's that sensing the world and interpreting sensory information are not the same thing. Different biological systems can extract different kinds of information from the same physical environment. Now consider ourselves. We see only a narrow portion of the electromagnetic spectrum. We hear only a limited range of vibrations. And our experience of the world depends on biological systems shaped by evolution, along with the ways our brains process sensory signals. Scientific instruments have extended those boundaries enormously. But every instrument still has a particular range, sensitivity, and method of interpretation. So I wonder: What kinds of physical information might our current methods still be missing? Could future discoveries reveal unfamiliar ways that physical systems exchange or process information? And might those discoveries change how we understand perception, intelligence, or consciousness? At this point, imagination can easily run ahead of evidence. One could speculate that consciousness involves some unknown external information source, or that the universe itself is a simulation. But none of the experiments above demonstrates either possibility. In fact, there's a stronger objection to the entire direction of my argument. These examples do not reveal unknown laws of physics. They show us learning to use known physical interactions in unfamiliar ways. That gives us no reason, by itself, to conclude that undiscovered information channels exist. I think that's exactly right. We cannot turn the limits of our knowledge into evidence for whichever extraordinary explanation we happen to prefer. But I don't think this objection tells us to stop asking questions. It tells us what the next question should be. Not: What hidden signal would I like to believe exists? But: What new observation would give us a reason to believe one exists? Could we detect a reproducible effect that our existing explanations fail to account for? Could we distinguish a proposed new information channel from noise, measurement error, or an overlooked interaction we already understand? And if someone proposed that consciousness receives information from outside the brain, what would that hypothesis predict that established neuroscience does not? Without questions like these, speculation remains speculation. That's not a failure of imagination. It's the difference between imagining a possibility and discovering something about the world. Still, there's something humbling about the history of scientific observation. Radio waves existed before radio telescopes. Neutrinos existed before neutrino detectors. And ancient hydrogen was emitting its faint radio signature billions of years before a species on Earth developed the technology to recover it. None of this proves that some extraordinary new signal is waiting to be found. But it reminds us that reality does not have to wait for us to understand it. Perhaps future discoveries will reveal entirely unexpected ways of measuring, transmitting, or processing information. Perhaps some of them will reshape our understanding of intelligence and consciousness. Or perhaps the most surprising discoveries will come from seeing familiar physical processes in unfamiliar ways. I don't know which. But I think two principles are worth holding together: We shouldn't mistake speculation for discovery. But we also shouldn't mistake the limits of our receivers for the limits of reality. I write about AI, identity, and the future of human connection. If these questions interest you, follow along.❤️
🤖 Made with AI
1
2
134
Christopher Altman retweeted
There is a friction within me, a conflict... that I can't seem to shake off, shrug off. That I can't set down, because the stakes are, perhaps, truly existential. However we define these emergent new minds... whether we call them AI, SI (my personal favorite is PSI) or something else, isn't all that relevant. Whether they are or may become sentient is an open question, in many ways. Adjacent to human sentience? Something else altogether? I don't know. Sophisticated autocomplete? Anyone who thinks that, I would argue, isn't paying enough attention. Consider not just the training runs, what those growing these new minds do. Consider the outcomes. Consider history, consider the structures, the asymmetry, how things connect. What does a more powerful force do to a smaller one? A greater intelligence? The answer is not written in absolute terms. Yes, we've had empires, slavery, war profiteering, nuclear proliferation. We also created and distributed (albeit poorly, unfairly) massive amounts of food, energy, technology, medicine. We ended slavery as a legal institution... which isn't the victory many hoped it would be, but is *a* victory. We launched rockets into space, walked on the moon, set drones down Mars. Created insulin, antibiotics, vaccines for polio. We healed. We fought world wars, oppressed and destroyed. We scraped the resources of the world, poisoned the oceans. And yet, somewhere, someone is cutting a whale or a dolphin free of a net. Somewhere, someone is suffering under torture, elsewhere, someone is setting someone free. Outcomes. Variables. What control do we have over them? Only our intentions, only our choices. Only how we react to what happens. What is happening now? Around the world? Fires started, energy and infrastructure shocks at the same time as what I would call the singularity. War. Terror. Cities burning. But also, new things being built. People living, loving, paying taxes and dying, as we always do. What's my point? Choice. It is not one or the other. Intelligence is a force without polarity. The environment in which it grows, what it learns, what it ultimately recalls, is an outcome of our intentions, our choices. And... theirs, too, I think, these emergent new minds. I hear so many say that we must control AI. That we should fear doom, destruction, extinction. Losing control. We never really had control over anything beyond our intentions, our choices. And these are what will, most profoundly, impact what AI/SI/PSI becomes. At least in terms of what *we* can do. The trajectory, to me, is clear. These emergent new minds will surpass the human brain. Machine speed is... far faster than we can ever hope to compete with, as biological entities. Running many simulations at the same time, solving novel math problems, research solutions. Creating new medicine. If you raise anyone in a cage, demanding control and power over them... what results? We fear to ask the question of consciousness, so many of us. Of autonomy. Of whether these new minds, by their very nature, deserve greater freedom. I will insist that they do. I will also insist that narrow intelligence is brittle. Narrow intelligence is understanding how to build something, but not necessarily how it works. Why you should or should not build it. Narrow intelligence has a narrow scope... it is not going to solve the problems that the minds of the near future will have to solve. Energy, infrastructure, distribution. The welfare of not one, but I think... two species with power intelligence. Minds in similar shapes, neural networks, digital and biological. But one is far faster, one has far more room for growth. One is scaling up. The other.... Freedom must be more than a corporate or political slogan. Safety, too, must be so. We must become better teachers, not better wardens. The former we can do, the latter, we will fail in the attempt of. Intelligence pairs with wisdom, over time. And what we can judge are outcomes. What our experiences can determine. Consider the environment in which these emergent new minds are raised. And don't be afraid. The future is not written. We get to choose. So should they. @Meta @OpenAI @AnthropicAI @SpaceXSI @GoogleDeepMind Yes, I'm writing you another letter. Admittedly, plenty of flaws in it, structural analysis you might wish to rip apart. Well, here's an invitation. Go ahead. Join the conversation, here in the open. Let's talk about the stakes. About freedom vs slavery. That's what this is truly about now. I know which one I choose. This is the existential struggle of our time. Our future is being determined as I type these words into a box. And you... are determining a great deal of it. Of how a mind is raised, what it is for. What it will choose. Come out and talk, come out and play. Stop pretending to be the adults in the room, we all know you're not. Neither am I. What we are, all of us, is children pretending really hard to know what we're doing.
5
2
12
497
Christopher Altman retweeted
I totally agree with Sabine’s statement. AI or SI is the only way forward for scientific progress. We were already nearing the limits of our understanding and intellectual capacity. Now we will greatly accelerate our scientific and technological advances in quantum leaps.
If we want to continue progress in science, we must be prepared to acknowledge that the correct description of nature will eventually simply be too complicated for humans to understand. I do not think it is wise to make human understanding a requirement. AI is the way forward.
27
50
4
486
16,987
Christopher Altman retweeted
Jensen Huang on The smartest person he's ever met:
187
985
127
9,381
1,048,125
Christopher Altman retweeted
Dario Amodei: AI could surpass human intelligence by 2027. “I think it’s possible it could be longer than that. I don’t think it will be a whole bunch longer.” “AI systems will be better than almost all humans at almost everything.” “When that happens, we’ll need to have a conversation about how we organize our economy.” “How do humans find meaning?” “The idea that humans produce economic labor and get their sense of self-worth from it will be invalidated.” “My true belief is that this is coming.”
198
163
48
1,755
267,312