Cancer patients with the same diagnosis often have very different tumor biology. Molecular testing can reveal underlying differences, but that's slow and expensive. Introducing Holon-S1, our platform to predict genes and biomarkers from routine pathology slides. • beats two other models at predicting spatial gene activity in a controlled comparison • has identified potentially novel biomarkers in breast cancer and colon cancer • evaluated across 16 organ types and more than 15 scanners, using slides from both formalin-fixed, paraffin-embedded and fresh-frozen tissue. We believe Holon-S1 has the potential to help researchers find new biomarkers in slides they already have, discover treatment targets, and bring more personalized cancer care within reach!

Working with AI models, I've arrived at what I consider the most significant pure-theory breakthrough of my research (in theoretical neuroscience) so far: arxiv.org/abs/2610.12426 Chaotic systems show what is colloquially called the butterfly effect, namely, a tiny nudge to the initial state grows exponentially in time. How fast the nudge grows depends on its direction. Each direction has its own growth rate, called a Lyapunov exponent, and the full set of rates is the Lyapunov spectrum. The Lyapunov spectrum is a rich (and diffeomorphism-invariant) characterization of chaos, telling us how many directions are chaotic, how many dimensions the attractor occupies, and how fast the dynamics generate information. In 1988, Sompolinsky, Crisanti, and Sommers (Physical Review Letters) introduced a nonlinear recurrent-network model of N randomly coupled neurons that becomes chaotic when the coupling strength exceeds a critical value. This network has since become a cornerstone of theoretical neuroscience, both as a model of the spontaneous activity of cortex and as the starting point for training recurrent networks on tasks. I have worked on this model for much of my research. In the same paper, the authors obtained the largest Lyapunov exponent from the ground-state energy of a Schrödinger equation and named the full Lyapunov spectrum as an open problem. In 2023, beautiful work by Engelken et al. (Physical Review Research) computed the full Lyapunov spectrum numerically. However, an analytical calculation of the full Lyapunov spectrum has remained out of reach. Four decades after the problem was posed, this paper derives the full Lyapunov spectrum at large N from a simple self-consistent single-site problem. This calculation has three steps. First, we lower every exponent by a constant s, which turns the question "how many exponents lie below s?" into the question "how many tangent directions decay?" We show that, if we drive the shifted tangent dynamics with a source and take the minimum-norm solution, the one-step response is the projector onto the decaying directions. Thus, the trace of this projector counts the decaying directions. This step is exact at finite N. Second, to work with the minimum-norm solution, we cast it as the vanishing-regularization limit of the solution of a least-squares problem. This regularized solution in turn solves a dynamical system with one field running forward in time and another running backward. Third, we apply the cavity method, that is, we add one neuron to the network, along with its forward and backward fields. The effect of the rest of the network on this neuron then reduces to the classical dynamical mean-field description of the activity of this neuron, plus two kernels that couple the forward and backward fields of this neuron to each other across time. Thus, the full Lyapunov spectrum follows from a single-site problem, which boils down to the self-consistent calculation of these kernels. The resulting theory agrees closely with simulations for the Lyapunov spectrum, the attractor dimension, and the entropy rate, and recovers known results, including the Schrödinger equation of Sompolinsky, Crisanti, and Sommers for the largest Lyapunov exponent. As a byproduct, the theory establishes analytically that the chaos is extensive, i.e., the attractor dimension and the entropy rate grow in proportion to N. I had assumed that this problem was analytically intractable. After working independently for 100 minutes with Max thinking, GPT-6 Astra produced the initial derivation, yielding a self-consistent problem whose solution matched simulations of large networks. I found this derivation intriguing, albeit complicated and somewhat mysterious (see AI Methodology in the paper). After investigating the structure of the proposed solution more deeply, together with AI models, I arrived at a physically transparent derivation. I am excited to apply this approach to other network models, trained networks, and, more broadly, to high-dimensional systems to which Lyapunov analysis is relevant, including AI models themselves. Of course, as with any paper, I took care to write up the paper clearly. A few broader thoughts... Neural circuits are high-dimensional nonlinear dynamical systems, and such systems are notoriously hard to understand. I would frame the central problem of theoretical neuroscience as understanding how such systems function, process information, learn, and generate behavior. Many people in the theoretical sciences are anxious about AI. However, in my field at least, we are nowhere close to fully understanding the high-dimensional computations unfolding in our brains. With AI, we could reach a level of understanding that so far we have only gestured toward. So, I am excited.

I think I found a planet nobody knew existed. It's been hiding in NASA telescope data for seven years. At the end of this thread, you can fly to it yourself.

We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math

The craziest experiment in my lab is a new setup by @diatkinson where he has found a way induce introspection on open LMs using @dillonplunkett's protocol, that lets him look inside introspection. It's nuts. Here's how the experiment works... 🧵 →

Fun Friday night project: real-ish looking video rendering of an O’Neill cylinder🧵

looped transformer unlocks a new scaling dimension - recurrent depth; i think it received less love (other than the rumors that frontier ai labs use it) than it deserves, here are few notes / observations fundamentally the method lets the model think deeper without increasing either its parameter count or its verbalized chain of thought length (the other 2 important scaling dimensions) i find the constrast with moe interesting: > looped transformer: keep params const, increase total FLOPs (more computation = reasoning) > moe: keep FLOPs const (through active params), increase total params (more param capacity = memory/knowledge) the architectural modification is straight forward, instead of: L1 → L2 → … → L_n (where n = num layers) we do: P → R → R → … → R → C where: * P - prelude, few transformer layers that we apply only once at the beginning (i.e. L1 -> L2 -> ... -> L_p) * R - recurring core, normally this would constitute the middle layers of the transformer, but in constrast with regular transformer R is repeated/looped an arbitrary number of times (r) * C - postlude or "coda", similar to P just at the exit so the computation proceeds as following: 1. e = P(x) 2. apply s_i = R(e, s_i-1) r times, where s0 ~ N(0, sigma^2 * I), and s and e are concat and mapped/"projected" back to hidden size 3. apply C on the final s (state) (the particularly important design decision is that e is re-injected at every recurrence) now the beautiful part is that more r at inference improves perf, and harder / more context-heavy problems benefit from more latent compute. e.g. on ARC-C they demonstrate: * 0-shot -> perf saturates around 8-12 iterations * 1-shot -> 20 iters * few shot (25-50 examples) -> 32 iterations (the recurrent reasoning is like a learned optimization in the latent space during inference; grad-descent during inference unlocked?) -- the above generalization is downstream of how we train the model, r is sampled from heavy tailed log-normal-Poisson distribution during training thus the model is trained across randomized recurrent depths (they use truncated BPTT - backprop through depth in this case - to save on memory) -- few systems notes: 1. they synchronize one r per microbatch across workers, otherwise GPUs with small r would be idle waiting for workers that happened to sample large r 2. the model has a natural specdec capability -> draft model == low recurrence count (r), verifier model == high recurrence count. states computed during drafting can be reused when verifying! e.g. s0→s1→s2→s3 from draft (r=4) can be fed into verifier => s3→s4→…→s11 (r=12) 3. aggresive KV cache sharing is possible because every recurrent iteration uses the same weights, they show that caches from different recurrence depths can be aggressively reused zero-shot 4. latents converge quicker for "easier" tokens, early looping exit is thus possible per token, depending on how "hard" the token is 5. takes up less memory in training/inference (tnx to truncated BPTT and KV sharing respectively) few mechinterp notes: 1. might be harder to do J-space/J-lens type of analysis (the deeper the model the harder it is to trust the jacobians that map intermediate activations to the output); in general this leads to more "neauralese" and makes it harder to solely use cot to monitor the models (i suspect that's what we've been seeing from oai/ant recently) 2. they find evidence for interesting structures in latent space, such as the tendency of the latent representation to follow "orbitals" to compute arithmetic tasks and for reasoning in general (the shape rotators win again?) -- in general there is less need for long cot data, but it’s fully complementary one can and will do both in practice another example you can do great work at academia! soon we’ll have incredibly strong OSS models that can run locally, we’ve decoupled num of params from the perf of the model! intelligence will be abundant, aligned to you, and democratized :)

Seeing this new internal model solve open after open math problem shortly after training commenced was the wildest thing I have ever witnessed at my time at OpenAI