
JEPA enjoyer @BrownUniversity, prev building world models @overworld_ai, data @Bloomberg 🇱🇧/🇺🇸
Providence, RI
Joined March 2014
- Tweets275
- Following602
- Followers474
- Likes9.7K
Sami retweeted
We're releasing an open-source collection of 80 simulation environments, and they're *fast* ⚡
stable-envs train WMs up to 30% faster, and with much better GPU utilization. Check it out!
arxiv: arxiv.org/abs/2611.23151
Scared for the future of open source? Black Friday discounts got you disappointed? NO MORE! With stable-envs:
- save 99% of disk
- rendering speedups of up to 20x faster than baseline
- train world models up to 30% faster end-to-end
- on 80+ of the most popular envs
- focus on training, modeling, and NOT data pipelines!
- open source! (see thread)🧵
We'd love to see you use it! If your environment is not supported yet, please open a PR!
This library was built by myself and my collaborators, @_fracapuano @ahitagnied , @_rwgao, and @randall_balestr ! Huge thanks to everyone involved as well.
5
129
Sami retweeted
Introducing LeVJEPA 1.5 Large, the first step in scaling our method.
It closes the gap to V-JEPA on both image and video benchmarks, trained with 5.6x less compute and over an order of magnitude less data.
Sami retweeted
The Platonic Representation Hypothesis is now a Theorem ◼️
Train JEPAs separately, on any modalities: they provably learn the same representation up to one rotation.
The result? A 109M JEPA-Text model that aligns with vision models better than 72B LLMs.
w/ @ylecun 🧵
Sami retweeted
Cancer patients with the same diagnosis often have very different tumor biology.
Molecular testing can reveal underlying differences, but that's slow and expensive.
Introducing Holon-S1, our platform to predict genes and biomarkers from routine pathology slides.
• beats two other models at predicting spatial gene activity in a controlled comparison
• has identified potentially novel biomarkers in breast cancer and colon cancer
• evaluated across 16 organ types and more than 15 scanners, using slides from both formalin-fixed, paraffin-embedded and fresh-frozen tissue.
We believe Holon-S1 has the potential to help researchers find new biomarkers in slides they already have, discover treatment targets, and bring more personalized cancer care within reach!
Sami retweeted
Introducing H-JEPA: the first end-to-end learned hierarchical world model for long-horizon visual planning!
Built atop SIGReg/LeWM, H-JEPA provides a stable training recipe where semantic abstractions naturally emerge across levels...
Paper+Code: arxiv.org/abs/2610.06805 🧵👇
Sami retweeted
Can a distributional loss pretrain a powerful one-step generative model from scratch?
The answer is YES but ONLY in the right feature space!
With the wrong representation (eg VAE features), training fails miserably and we uncover why in this work accepted to Neurips2026!
🧵
Sami retweeted
I was encouraged this week to see the leaders of the frontier labs agree on the need for them to slow down the pace of AI development. Given the stakes, it’s a good and necessary first step.
But I’m even more encouraged by the growing recognition that how this powerful new technology develops should be at the center of our public debate.
I’ve been watching the progress on AI for over a decade now, and one thing that’s clear to me is that the potential impact of this technology is not overhyped. It’s also moving at lightning speed – and even faster than those who are engineering it can keep up with.
I’m not an AI accelerationist who believes it will lead to some techno-utopia, and I’m not a doomer who thinks it will inevitably lead to humanity’s destruction.
But whether this technology results in amazing breakthroughs in medicine, energy and education or unleashes huge economic disruptions, greater inequality, and potential catastrophe will depend on the choices that we make right now – choices that should be made not just by the companies involved, but by all of us.
Sami retweeted
If you missed our opening remarks:
- 4th world modeling workshop: Aspen, CO, February, world models for physics, wmw-aspen.github.io
- 1st world modeling conference: Bay Area, CA, May, icwm.cc
In collaboration with @ylecun @LambdaAPI @amilabs and more TBA!
Sami retweeted
SIGReg for pretraining Video Foundation Models! Our LeVJEPA opens many doors...
- stable recipe with a simple loss (sigreg + prediction)
- no tubelet, frame aggregation, EMA, stop-gradient, ....
- 20X more FLOP efficient than VJEPA1/2 pretraining
- open source + reproducible
A new Pareto frontier in video pretraining.
Excited to introduce LeVJEPA 🔥: a stable, efficient end-to-end pretraining method that matches V-JEPA 2 at up to 20x less pretraining compute!
No target encoder, no masked prediction, no stop-gradient or teacher-student schedule.
One encoder, trained with a single objective. 🧵
Sami retweeted
Replying to @JitendraMalikCV
Exactly.
We also should not confuse world models, as per your definition, with video prediction or video generation models.
Understanding the dynamics of a system in order to control it is not the same as producing cute videos.
Sami retweeted
We are two weeks away from the third World Modeling Workshop!
- check wm-booth.org/ for all the latest info/links
- livestream and recordings will be available for free to everyone, no registration required
- a few surprises will be announced during the opening remarks
Sami retweeted
Action chunking is a mysteriously effective method. Modern large-scale imitation learning basically doesn't work without it. But why does it actually help? In our new paper, we try to break down the reasons. As the saying goes, what happened next might surprise you...
Action chunking is a critical component in virtually all modern approaches to imitation learning for robotics.
But why is it so critical, and do we really need action chunking? Check out our latest work to find out! (1/n)
action-chunking.github.io
Sami retweeted
We made an Elden Ring boss fight playable inside a real-time video world model.
Text is not a scene prompt. It is the controller.
Play Incantation:
reactor.inc/incantation
Sami retweeted
Today, I will give a talk about my recent Latent Action Model paper, including one possible recipe for for training Latent Action Models to learn Super Mario 🍄🕹️ from observation alone. Recordings will be available later on the same platform 🎥
Heejeong (Hazel) Nam of @BrownUniversity presents a novel approach that learns reusable motion primitives for more robust action understanding in AI.
🎙️ Hosted by @ceciletamura, Head of Community, @PloutosDev
📅 July 27, 2026
🕓 4:00 PM PDT
🔗 ploutos.dev/?stream=calm-ind…
Sami retweeted
Our humble recommendation- there is at least one better approach than EMA for latent space training arxiv.org/abs/2509.24317
why does JEPA/DINO-style training produce object-level structure in attention maps with zero segmentation supervision? one specific mechanistic hypothesis keeps coming up: the EMA teacher itself is doing a lot of the actual work
DINO's own ablation (Caron et al, arxiv.org/abs/2104.14294) is the cleanest evidence for this: swap the momentum-updated teacher for one that just copies the student's weights directly, and training fails to converge, while if you keeping the EMA, object boundaries emerge in the attention maps on their own, no segmentation labels anywhere in the loop
I-JEPA and the rest of the JEPA family inherited the same EMA teacher design for the same reason
my read on why this might matter mechanistically: a teacher that changes slower than the student acts as a moving but stable target, so the student can't just chase whatever the teacher did last step, it has to find features that stay predictive across many steps of teacher drift (objects are exactly the kind of feature that stays stable under small transformations while background noise doesn't, so a slow-enough target might be implicitly selecting for object-level structure rather than being told what one is)
still an open question how much of this is EMA specifically versus any sufficiently slow, sufficiently stable target (haven't found a paper yet that isolates the two cleanly)
Today's leading SSL methods rely heavily on training heuristics—EMA, teacher-student training, layer freezing, and more—to remain stable.
Regularization-based methods are much simpler, but have long failed to match the generalization performance of leading SSL approaches.
VISReg breaks this trade-off.
We introduce VISReg, the first regularization-based SSL method to achieve state-of-the-art out-of-domain generalization, outperforming MoCov3, DINO, data2vec, iBOT, I-JEPA, MAE, and DINOv2.
arXiv: arxiv.org/pdf/2606.02572
Github: github.com/HaiyuWu/visreg






















