i built an embedding engine by composing vectors from common abstraction neuron activations (h/t @nickcammarata "already clean" essay)
and it behaves more consistently especially when doing embedding math. less distracted by spelling/formatting and more focus on abstractions present
it's not always better, but generally at least on par. i have not seen it breakdown into returning junk like regular embedding models do yet. i am gonna try running some vector search evals on it to see how it performs
reproduced very well. qwen 4b forgot facts but can still do reasoning (this is the 99% variant)
cammarata skeptics in complete shambles as they ramble with theory and composure while people keep applying the ideas in the essay to real work with undeniable results within days. and this isn't even the only one
murat 🍥 retweeted
Inspired by Nick Cammarata’s interpretability essay “Neural Networks Are Already Clean”, I turned a couple of its ideas into speedups for pretraining without backpropagation. Still in the works. 2.3x speedup, cool xitter.cf/industriaalist/status/…
Replacing backprop with zeroth-order optimization is inevitable.
Introducing Noise: a speedrun for pretraining without backprop.
Last week we presented Dust, the first method to approach backprop with SGD for pretraining transformers. We think another 1,000x improvement on top of it is possible, and it will require *co-designing* credit assignment, optimizers, architectures, and eventually, hardware.
We built Noise to iterate on this faster.
> gradient ascent to visualize neurons was misleading
> interesting to think about gradients in general, how cleanly can the derivative represent recursive generalizations
> feature breakdown circuits: new type of embeddings possible with some effort?
> i have ideas
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
murat 🍥 retweeted
since people seemed to enjoy our locomotion results, chef 🧑🍳@octi_zhang cooked up a special treat à la SGS on a different embodiment (a wheeled @UnitreeRobotics robot) to celebrate the release! The robot is placed in this big world, and its job is to reach the yellow dots.
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before.
The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices.
Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL.
No demonstrations, no per-task reward tuning. Just PPO at scale.
Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
i dont think turn-based RL could ever solve the tail end of broken software. perceiving the result of actions IN REALTIME is a requirement. digital optimus or thinky interaction model seem like more serious long term agi projects than chatbot gigahacker coding RL environments to me
they won't be as superintelligent on jagged domains but they're needed to close the gap on generality
teh term "system 1" being taken up by decision models is a linguistic shame because system 1 is for realtime perception and reaction not some category classification task
murat 🍥 retweeted
JEPA is one of the best ideas in how machines can learn about the world, however, it is still under-explored which parts of the state it learns stay the same and which parts change.
For example, a coffee mug looks different in bright morning sunlight and in dim light at night, but my hand still reaches for the same spot, because my brain keeps where the mug is separate from the lighting, which changes but does not matter for reaching.
Our new paper, SplitJEPA, gives JEPA this same separation in the state it learns.
In fact, the LeJEPA line of work from @ylecun, @randall_balestr, and @klindt_david already proves something strong here. Under stationary Gaussian predictive dynamics, they show that latent prediction with Gaussian regularization recovers the complete latent state up to a global orthogonal transformation, with no reconstruction needed at all.
That remaining transformation is where our paper begins.
Picture a map where every distance between cities is correct but there is no compass. Technically, nothing is missing, yet you still cannot tell which way is north.
For LeJEPA, any orthogonal transformation of the learned state is an equally optimal solution, so each dimension of the learned representation can be a linear combination of invariant factors, which are shared across related observations, and variant factors, which change between them.
SplitJEPA adds that compass. It divides the representation into an invariant block and a variant block, and alongside the predictive pairs it also trains on invariant pairs, where the two observations in a pair share the invariant factors and differ in the variant factors, such as the same physical state rendered from two cameras.
An agreement loss requires the invariant block to match across each pair, which removes its linear dependence on the variant factors. Because the full transformation is orthogonal, once the invariant block holds exactly the invariant factors, the variant block is left with exactly the variant factors.
The key condition is sufficient variation. SplitJEPA requires the second-moment matrix of the variant changes across pairs to be full rank, meaning no variant factor, and no combination of them, remains unchanged across all pairs. A single pair does not need to change every factor though, since the condition is on the aggregate coverage.
We prove that under these conditions the invariant and variant subspaces are identified up to independent block-wise isometries, without introducing an observation decoder.
The result is also stable. When the agreement holds only approximately, the leakage of variant factors into the invariant block is bounded in terms of the consistency error and the smallest eigenvalue of the variation matrix.
We evaluate SplitJEPA on PushCube, a ManiSkill simulation task where a robot arm must push a cube into a goal region on a table.
When the test camera is rotated 10 degrees around the scene (a camera pose not seen in training), behavior-cloning policies succeed only 12.8% of the time on raw pixels, and 48.0% on frozen R3M (a pre-trained visual representation for robot manipulation), while SplitJEPA achieves a 71.5% success rate.
LeJEPA showed that the latent state can be recovered without reconstruction, and SplitJEPA extends this to show that its invariant-variant organization can, and should, be recovered the same way.
Replying to @Helios_Hua
The key idea of SplitJEPA is to combine predictive learning with matched observations that share invariant factors but differ in variant factors, using their complementary information to resolve latent block ambiguity.
murat 🍥 retweeted
Oh wow this is actually really cool!! Although I believe ame2 can also solve this type of terrains without expert policies distillation, achieving this by rather a simple modification on ppo without massive reward tuning is quite impressive.
Replying to @mateoguaman
Our manipulation results were mind-blowing the first time I saw them, but earlier on, my favorite result was this locomotion run. Many great people have done parkour-type skills with locomotion already, but as far as I am aware, most, if not all, of these works trained separate policies for each terrain, and then distilled into a single policy. They also typically contained tons of hand-engineered rewards and curriculums.
A very early motivation for SGS was to simply figure out if we could get similar results but get rid of as much of this hand-engineering as possible, and we managed to achieve that! This video shows a continuous 1-min run in much harder terrains than ever before, all solved by the same policy.
murat 🍥 retweeted
🎯 Learning is most effective at the frontier of capability: problems that are too easy or too hard teach nothing.
For LLM reasoners trained with GRPO this is literal: problems the model always or never solves give zero gradient.
We introduce Frontier Learning👇🧵
intuitively makes sense, by rewriting training data to optimize for surprise only where it matters the backprop is given much cleaner signal. with this line of thinking one could expect GRPO to not work that well when using N human provided samples instead of on policy generated



























