Vancouver, BC
Joined March 2008
reproduced very well. qwen 4b forgot facts but can still do reasoning (this is the 99% variant)
Tested this! I flattened every neuron's weakest activations Overall loss barely changes at first but the models get REAL bad at memorized text. So the weak tail does, as Neel expected, contain memorized things. Weaker activations seem to correspond to less generalizable content
1
5
709
im going, come have a pint or four
OFFICE HOURS SAN FRANCISCO WIZARD OF FRIENDLINESS EDITION!!! Drink booze! Eat food! Talk about that one movie you saw! Zeitgeist, after 6. Look for the duck! MEET YOUR MUTUALS! (AND EVERYONE ELSE) YOU'RE YOUR OWN WIZARD
1
1
5
1,297
cammarata skeptics in complete shambles as they ramble with theory and composure while people keep applying the ideas in the essay to real work with undeniable results within days. and this isn't even the only one
forward-only pretraining now matches backprop on the Noise benchmark. Relay: val 3.704 in 2.5 min on 8×H100. backprop with the same model, data and optimizer: 3.712. the current record, Boost→Cut: 3.945 in 7.6 min. no autograd anywhere. 🧵
8
8
297
16,129
Inspired by Nick Cammarata’s interpretability essay “Neural Networks Are Already Clean”, I turned a couple of its ideas into speedups for pretraining without backpropagation. Still in the works. 2.3x speedup, cool xitter.cf/industriaalist/status/…
Replacing backprop with zeroth-order optimization is inevitable. Introducing Noise: a speedrun for pretraining without backprop. Last week we presented Dust, the first method to approach backprop with SGD for pretraining transformers. We think another 1,000x improvement on top of it is possible, and it will require *co-designing* credit assignment, optimizers, architectures, and eventually, hardware. We built Noise to iterate on this faster.
28
71
20
1,302
160,394
> gradient ascent to visualize neurons was misleading > interesting to think about gradients in general, how cleanly can the derivative represent recursive generalizations > feature breakdown circuits: new type of embeddings possible with some effort? > i have ideas
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
2
23
1,589
this is more than just alignment stuff
1
6
521
more importantly if the weights are in fact so clean this is a nuclear bomb for continual learning but it will take a few more insights to get there if indeed possible
5
283
since people seemed to enjoy our locomotion results, chef 🧑‍🍳@octi_zhang cooked up a special treat à la SGS on a different embodiment (a wheeled @UnitreeRobotics robot) to celebrate the release! The robot is placed in this big world, and its job is to reach the yellow dots.
(1/5) Everyone says dexterous manipulation needs real-world data. These policies were trained with none. Our robots mesh gears, thread nuts, and insert pegs zero-shot from RGB, and the same recipe works for locomotion across terrains much harder than before. The missing piece wasn't a better RL algorithm, but a small tweak to what the robot practices. Today we're releasing Success-Guided Sampling (SGS), a simple change to the task sampler that unlocks new levels of dexterity and agility with large-scale, end-to-end sim2real RL. No demonstrations, no per-task reward tuning. Just PPO at scale. Check out the policies yourself at sgs-rl.github.io! All videos at 1x speed! 🧵 🔈
4
14
1
105
4,549
i dont think turn-based RL could ever solve the tail end of broken software. perceiving the result of actions IN REALTIME is a requirement. digital optimus or thinky interaction model seem like more serious long term agi projects than chatbot gigahacker coding RL environments to me they won't be as superintelligent on jagged domains but they're needed to close the gap on generality teh term "system 1" being taken up by decision models is a linguistic shame because system 1 is for realtime perception and reaction not some category classification task
I tried the adobe rip offs, they’re so bad I will just continue paying adobe just pure garbage doesn’t even work
1
3
22
2,013
murat 🍥 retweeted
JEPA is one of the best ideas in how machines can learn about the world, however, it is still under-explored which parts of the state it learns stay the same and which parts change. For example, a coffee mug looks different in bright morning sunlight and in dim light at night, but my hand still reaches for the same spot, because my brain keeps where the mug is separate from the lighting, which changes but does not matter for reaching. Our new paper, SplitJEPA, gives JEPA this same separation in the state it learns. In fact, the LeJEPA line of work from @ylecun, @randall_balestr, and @klindt_david already proves something strong here. Under stationary Gaussian predictive dynamics, they show that latent prediction with Gaussian regularization recovers the complete latent state up to a global orthogonal transformation, with no reconstruction needed at all. That remaining transformation is where our paper begins. Picture a map where every distance between cities is correct but there is no compass. Technically, nothing is missing, yet you still cannot tell which way is north. For LeJEPA, any orthogonal transformation of the learned state is an equally optimal solution, so each dimension of the learned representation can be a linear combination of invariant factors, which are shared across related observations, and variant factors, which change between them. SplitJEPA adds that compass. It divides the representation into an invariant block and a variant block, and alongside the predictive pairs it also trains on invariant pairs, where the two observations in a pair share the invariant factors and differ in the variant factors, such as the same physical state rendered from two cameras. An agreement loss requires the invariant block to match across each pair, which removes its linear dependence on the variant factors. Because the full transformation is orthogonal, once the invariant block holds exactly the invariant factors, the variant block is left with exactly the variant factors. The key condition is sufficient variation. SplitJEPA requires the second-moment matrix of the variant changes across pairs to be full rank, meaning no variant factor, and no combination of them, remains unchanged across all pairs. A single pair does not need to change every factor though, since the condition is on the aggregate coverage. We prove that under these conditions the invariant and variant subspaces are identified up to independent block-wise isometries, without introducing an observation decoder. The result is also stable. When the agreement holds only approximately, the leakage of variant factors into the invariant block is bounded in terms of the consistency error and the smallest eigenvalue of the variation matrix. We evaluate SplitJEPA on PushCube, a ManiSkill simulation task where a robot arm must push a cube into a goal region on a table. When the test camera is rotated 10 degrees around the scene (a camera pose not seen in training), behavior-cloning policies succeed only 12.8% of the time on raw pixels, and 48.0% on frozen R3M (a pre-trained visual representation for robot manipulation), while SplitJEPA achieves a 71.5% success rate. LeJEPA showed that the latent state can be recovered without reconstruction, and SplitJEPA extends this to show that its invariant-variant organization can, and should, be recovered the same way.
Replying to @Helios_Hua
The key idea of SplitJEPA is to combine predictive learning with matched observations that share invariant factors but differ in variant factors, using their complementary information to resolve latent block ambiguity.
6
26
3
318
12,431
murat 🍥 retweeted
Oh wow this is actually really cool!! Although I believe ame2 can also solve this type of terrains without expert policies distillation, achieving this by rather a simple modification on ppo without massive reward tuning is quite impressive.
Replying to @mateoguaman
Our manipulation results were mind-blowing the first time I saw them, but earlier on, my favorite result was this locomotion run. Many great people have done parkour-type skills with locomotion already, but as far as I am aware, most, if not all, of these works trained separate policies for each terrain, and then distilled into a single policy. They also typically contained tons of hand-engineered rewards and curriculums. A very early motivation for SGS was to simply figure out if we could get similar results but get rid of as much of this hand-engineering as possible, and we managed to achieve that! This video shows a continuous 1-min run in much harder terrains than ever before, all solved by the same policy.
3
4
147
13,505
i tried outchilling this duck and got mogged (in 4k)
1
1
23
1,263
predictions 2 and 3 hit, still waiting on interpretable embeddings
Today, we are releasing a research preview of Rho-1, our 19B omni model that understands and generates text, images, video and robot actions, all inside a single neural network.
2
2
33
2,895
mining a block 2026 style
Full VM escape zeroday (guest>host root in industry standard hypervisors)! More soon
1
1
21
2,211
murat 🍥 retweeted
🎯 Learning is most effective at the frontier of capability: problems that are too easy or too hard teach nothing. For LLM reasoners trained with GRPO this is literal: problems the model always or never solves give zero gradient. We introduce Frontier Learning👇🧵
30
68
10
1,061
103,569
intuitively makes sense, by rewriting training data to optimize for surprise only where it matters the backprop is given much cleaner signal. with this line of thinking one could expect GRPO to not work that well when using N human provided samples instead of on policy generated
SFT is not dead! 🥳 We found a way to make SFT rival current prevailing posttraining methods, often generalizing better and forgetting less than RL and OPSD. 🤯 Following our prior work on reasoning with sampling, we now introduce sampling to the posttraining stack. 1/n
5
41
4,412
not just for creatives.. the selfish channeling of impulses and selfish intrigue in own perception , makes people very interesting and charming
favorite reminder for creatives, by Björk
6
51
3,132
in nyc for a bit
4
794
murat 🍥 retweeted
solving mechinterp and making embedding search work reliably are the same task
3
1
15
1,924