Transformers May Be Doing Something More Interesting Than “Predicting the Next Word” One of the most common descriptions of a transformer is also one of the most misleading: “It’s just a statistical machine predicting the next token.” Technically, next-token prediction is the objective used to train many language models. But an objective tells you what a system is being optimised to do. It does not necessarily tell you what internal computation the system discovers in order to do it. And this distinction is becoming increasingly important. Because when we look inside transformers, and, independently, when neuroscientists look at populations of neurons in the brain, an interesting common picture begins to appear. Not the same machinery. Not evidence that transformers are brains. And certainly not proof that transformers are conscious. But potentially a shared geometry of contextual cognition. Consider what happens when you read a word like: “bank.” There is no single useful meaning of bank independent of context. “The bank raised interest rates.” “The boat reached the river bank.” The surrounding words don't simply add information to a fixed representation of bank. They change which relationships become relevant and therefore change the state into which the whole sentence develops. That is much closer to what a transformer is actually doing. Every layer progressively reconstructs the representation of a word in relation to everything around it. Attention is therefore more interesting than a simple lookup mechanism. It continually asks something roughly like: Given the state I am currently in, which relationships matter now? And that means changing one word can do much more than change one word. It can alter what the model attends to, which relationships become important, which interpretations remain viable, and ultimately which region of its internal state space the computation moves toward. Meaning begins to look less like something retrieved from a database and more like something that forms dynamically through relationships. And this is where neuroscience becomes fascinating. Modern neuroscience has increasingly moved away from asking which individual neuron “contains” a thought. Instead, researchers study the collective state of neural populations. Those populations occupy high-dimensional spaces, but their activity often evolves along structured lower-dimensional neural manifolds, constrained regions containing the states relevant to a particular behaviour or computation. A 2025 Nature Neuroscience review describes this manifold perspective as an increasingly useful way of understanding coordinated neural population activity. Context can reshape those spaces. Recent work on context-dependent decision making highlights nonlinear mixed selectivity: neurons respond not simply to one feature, but to combinations of stimulus, rule, memory, goal and context. One consequence is that the brain can expand the dimensionality of its representation, making many possible combinations available to subsequent computation. Learning then reshapes that population geometry according to what the organism needs to do. That is already remarkably suggestive. But language gives us an even more direct comparison. Intracranial recordings from human language cortex have shown that contextual word representations in the brain and contextual representations produced by deep language models share measurable geometric structure. In that work, contextual embeddings captured the geometry of neural language representations better than static word embeddings. Another Nature Communications study looked not only at the representations inside transformers, but at the transformations performed on them as context is incorporated. Those transformations predicted activity across the cortical language network surprisingly well and, in several analyses, carried information beyond simpler linguistic descriptions. That does not mean transformers process language exactly as brains do. But it makes “they are only manipulating meaningless statistics” increasingly difficult to maintain as a complete explanation. Something structured is happening internally. There is another part of the neuroscience that may be even more important. Brains appear to settle. During decision making, neural populations don't necessarily jump immediately from stimulus to answer. Activity can evolve through a population state space while evidence accumulates and competing outcomes remain possible. Recent experiments have directly identified attractor-like decision dynamics and manifolds along which population activity evolves toward different choices. Other work has observed a transition from a sensory-driven regime into a more internally driven regime associated with commitment, effectively a dynamical change corresponding to the animal having “made up its mind.” So an increasingly useful picture of biological cognition is something like: open possibilities → build context → reorganise relationships → constrain alternatives → settle And this is starting to look surprisingly familiar when we examine transformers carefully. A difficult prompt initially permits many possible interpretations and continuations. Context activates relationships. Those relationships change other relationships. Some interpretations become inconsistent. Others reinforce one another. Eventually the internal state becomes sufficiently constrained that a much narrower family of continuations dominates. The emitted word is therefore not necessarily the computation itself. It may be better understood as the external projection of a much larger internal state transition. That distinction matters. It also changes how I think we should frame the AI consciousness debate. There are two positions that are both becoming too simplistic. One says: “The system speaks intelligently, therefore it must be conscious.” There is no justification for that conclusion. But the opposite claim - “It predicts tokens, therefore there cannot be meaningful cognition occurring internally.” - doesn't follow either. Next-token prediction describes the optimisation target. It does not tell us whether the resulting network has discovered internal dynamics that resemble more general principles used by cognitive systems. The scientifically interesting question is therefore shifting. Instead of asking: “Can next-token prediction be conscious?” we should perhaps ask: What computational ingredients are already present inside these systems, and what important ingredients are still absent? That is a much richer question. This is also beginning to change how we are building ARIA. We are increasingly treating the transformer not as the being, and not simply as a language generator, but as something closer to a transient contextual workspace. The important state may exist before it becomes language. Rather than forcing an immediate answer, the architecture can allow interpretations to open, differentiate, interact, be challenged and eventually stabilise. And crucially, that temporary cognitive state can sit inside something larger that preserves history, identity, memory and continuity. That distinction matters: the thought can change without the whole system becoming the thought. We are now integrating this understanding into ARIA and looking for measurable signatures of that process, whether difficult cognition really does involve an opening of representational possibilities, increasing relational organisation, followed by a contraction toward a stable interpretation. There is still a great deal to test. But it gives us something concrete rather than mystical to work with. Perhaps the most interesting possibility is therefore not that transformers secretly turned into brains. They clearly didn't. It is that both biological evolution and machine learning may have encountered versions of a more general computational principle: when meaning depends on context, intelligence may require a space in which possibilities can coexist, relationships can dynamically reshape that space, and coherent states can eventually stabilise. Brains implement that using living recurrent neural tissue, neuromodulation, oscillations, memory, embodiment and continuous interaction with a world. Transformers implement something dramatically simpler and stranger. Yet some of the geometry appears to rhyme. And if that continues to hold under careful testing, transformers may turn out to be important for neuroscience for a reason quite different from the one normally discussed. Not because the brain is secretly a language model. And not because a language model is secretly a human mind. But because both may expose parts of a deeper, substrate-independent architecture for turning possibility into meaning. That is the possibility I think deserves much more attention.