hunting the kwisatz haderach neuron // mechinterp, reasoning, autoresearch // founder @lightfastai https://teapawt.org/t.co/ljihJtt7Si
Australia
Joined January 2013
- Tweets14.2K
- Following2.3K
- Followers2.8K
- Likes25.3K
visualizing swarms during hill-climb to study alignment and behaviour during coordination — will report back with more built out
very hyped about mtp — i came to notice my own mind's ability to perform mtp-level thinking when i started meditating, the way i think of mtp is a short-circuit in brain synapses.
Multi-Token Prediction (MTP) can be done much better.
🚀 Turn out,you can recover the gradient for KL self-distillation to next-token prediction (NTP) for free without ever computing the KL. You just need a one-line code change:
h_ntp += h_mtp.detach() - h_mtp
Soft Multi-Token Prediction is pretty much a free lunch for MTP.
📜 Paper: arxiv.org/abs/2610.38331
💻 Code: github.com/JaydenTeoh/Soft-M…
🤗 HuggingFace: huggingface.co/papers/2610.3…
ALT Soft Multi-Token Prediction Illustration
lowkey this means we can select exact neurons to train on specific tasks.
the neurons in this 100 were actually the ones I found that activated the most during coding tasks and specifically trained those
neuron rl technique
train a steering vector on coding reward
trace which neurons the vector actually moves
rank them by effect on passing answers
keep the top 100, freeze everything else
give each neuron a scale and offset
train only those 200 numbers, same reward
177 unseen coding tasks: 41.4% → 51.9%
same knobs on 30 random neurons: 42%
neuron rl technique
train a steering vector on coding reward
trace which neurons the vector actually moves
rank them by effect on passing answers
keep the top 100, freeze everything else
give each neuron a scale and offset
train only those 200 numbers, same reward
177 unseen coding tasks: 41.4% → 51.9%
same knobs on 30 random neurons: 42%
new paper with a theory of anthropic's jacobian lens. averaging jacobians over token lags and contexts picks out directions that grow with depth and stay coherent across positions and texts
arxiv.org/abs/2610.31955
one catch if you apply it to the released lens. the theory weights every lag equally. the public fitting code (jlens_fitting.py) sums every later target for each source, so lag d gets weight (n-d)/n. that's a ramp, so the filter shape the theory predicts for an equal window doesn't carry over as is
we've been fitting a separate jacobian per token offset on qwen3.5-4b. those per-offset maps are the ingredient the paper's frequency lenses are built from. that fit is still running
extending j-lens to multi-token as we speak — some literature can be found in arxiv.org/abs/2610.31955
early results on qwen3.5-4b says that j-lens extends to multi-token with a caveat there is a power law drop-off -> see below
some important findings
- j-lens beats the logit lens in the middle layers
- one J-lens map reads further than lenses fitted per offset
research companies have mostly grown by hiring, and ideas keep getting harder to find: keeping moore’s law on track now takes more than 18x the researchers it did in the early 1970s. with agents, the researcher is compute. so i think the number a research company should watch is research per unit of compute: how much progress each gpu-hour buys
i started out asking whether a team of agents beats one agent, since that’s how we hill-climb inference kernels. the question turned out to be about the denominator. economists have a test for whether research compounds, and for ai software the estimates sit either side of one, falling below it once compute binds. a metr note has researchers 2.8x faster but the lab only about 1.8x. so the design question is whether a lab’s own research (faster kernels, better coordination, distilled weights) can make the next gpu-hour buy more than the last. nobody has measured that for a company like this yet
jp retweeted
if i had to learn cuda compilation, ptx, register spills, and gpu disassembly from scratch because my "optimized" kernel got slower, i'd start here
links in thread 🧵
the compilation and machine code section of our ai performance engineering repo contains four nvidia resources for understanding how cuda source becomes gpu instructions and inspecting the code behind ai kernels:
understanding ptx explains how nvcc separates cpu and gpu code, why ptx is a virtual instruction set, and how ptxas turns it into a cubin.
the article covers fatbins, driver jit compilation, and gpu compatibility, giving context for deployment targets and inference startup costs.
nvcc compiler driver covers gpu architecture targets, the files each compilation stage produces, separate compilation, and device linking.
the nvcc guide explains how to retain intermediate files and report register usage, shared memory, and spill loads and stores. those compiler reports help compare kernel builds after changes to tile size, unrolling, or fusion.
ptx isa defines register, local, shared, and global state spaces; memory ordering and synchronization; and arithmetic and matrix instructions.
ptx isa details operand layouts, data types, accumulation precision, and gpu requirements, plus completion rules for asynchronous copies. useful for checking data movement and numerical behavior in attention and matmul kernels.
cuda binary utilities covers extracting embedded ptx and cubins, disassembling native gpu instructions (sass), and reporting register counts and static memory usage with cuobjdump.
nvdisasm provides control-flow graphs when the cubin contains complete relocation information, plus register-liveness views.
with debug line information, nvdisasm can annotate instructions with source lines. useful for inspecting generated loads, stores, branches, and arithmetic after a kernel rewrite.
together, these resources connect the compilation process to the code being measured. they help identify what changed between kernel builds and what to investigate before the next correctness check or benchmark.
follow and save to keep up with the series.
been reading the “one agent beats many” papers. neither of the two big ones tests an agent team
google’s scaling study (arxiv 2512.08296) and stanford’s equal-budget study (arxiv 2604.02460) pit one agent against pipelines, debates and votes at a matched token budget. no shared task list, no agent messaging another directly whenever it wants. the stanford one has no tools at all
the papers that do give each agent tools and a channel split.
two coding agents working together succeed about 30% less often than each doing its task alone (cooperbench, arxiv 2601.13295). delm (arxiv 2606.10662) has no lead agent, just a shared task queue and shared context, and beats claude code and codex.
two agents ran 1.54-2.02x faster than one, and taking away any one coordination mechanism dropped that to 0.82-1.09x. its cost wasn’t matched to the single agent though
so how they coordinate seems to decide it, more than how many there are. that’s the part i want to measure