Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva) header
Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva) profile picture

Postdoc @Mila_Quebec @UMontreal @IVADO_Qc Like to explain unexpected behavior of neural nets 🤯

Montréal, Québec
Joined December 2018
Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)Photo by Ekaterina Lobacheva @ ICML 2026 (@KateLobacheva)
🚨 New paper alert: Loss Smoothing for Stable Adaptation Under Distribution Shift 💡Don't switch objectives abruptly — anneal from source to targe. This preserves useful features and improves adaptation in fine-tuning, CL, and offline-to-online RL. Thread by @dapatil211 👇
Excited to share our latest work: Loss Smoothing for Stable Adaptation Under Distribution Shift. TL;DR: When adapting a model, don’t abruptly switch to the new objective. Gradually smooth the transition from the old objective to the new one. Preprint: arxiv.org/abs/2607.00634 🧵 1/n
1
3
12
2,363
Ekaterina Lobacheva @ ICML 2026 retweeted
1/ Can we stop models from learning subtle biases in data simply by using regularization? Nope! Regularization amplifies subliminal learning and other forms of covert influence.
4
24
7
130
9,185
Ekaterina Lobacheva @ ICML 2026 retweeted
A science of post-training can be built on interpretability. Models know when they reward hack, and we see it, yet training ignores it. We propose Mechanistic Shaping: using the model’s own concepts to write training rules. In vision, DPO & RL, and its effects on monitoring 🧵
29
160
8
961
65,371
Ekaterina Lobacheva @ ICML 2026 retweeted
Looped LLMs are everywhere right now! But why are they interesting? Do they model language differently than regular LLMs? Grown and looped transformers are better sequence models! With @kaitlinmaile & @oswaldjoh (@Google, Paradigms of Intelligence Team) and @stefanAbauer. 🧵
8
27
3
142
8,193
Ekaterina Lobacheva @ ICML 2026 retweeted
Neural network weights are a novel modality in machine learning - that's amazing! "Weight-space programming" shows its potential for editing other neural networks' weights. But there is more. These are the slides from my keynote at the @icmlconf Workshop on "Weight-Space Symmetries" providing an overview of this research area: drive.google.com/file/d/1yy6… Thanks to the organizers for inviting me: @yanii, @BorisAKnyazev, @KateLobacheva, @adnan_ahmad1306, @orvieto_antonio, Alexander Theus.
5
18
5
163
381,597
Ekaterina Lobacheva @ ICML 2026 retweeted
⁉️⁉️ What if the harmful data we delete is exactly what makes AI safe? 💬 LLMs have quietly become the first place people turn when something is wrong: a symptom, a debt, a legal problem, a crisis. At this scale, AI safety is no longer an abstract research problem. It touches real lives, every day. 🚫 Yet most alignment today works by erasure: filter out unsafe data and train the model to refuse. The result is often a blanket "Sorry, I can't help with that." ⚠️ But refusing is not the same as being safe. A refusal leaves people with nothing to act on, and it can push them toward riskier, unmoderated sources exactly when they need guidance most. 💡 Here's the irony: unsafe data is abundant and packed with real domain knowledge, while safe, genuinely helpful answers are scarce and costly to curate. And we throw the abundant part away. 🤔 So we asked a counterintuitive question: can unsafe data actually make models safer? 🎉 Our answer is yes, and I'm thrilled to share that this work has been accepted at #NeurIPS2026! 📄 Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing
2
15
5
37
2,706
Ekaterina Lobacheva @ ICML 2026 retweeted
New preprint: Cross-modal steering vectors transfer zero-shot in both directions 🖼️ ↔️ 📝 From text generation to image generation and vice-versa Exciting evidence for a shared conceptual space! Fantastic effort led by Donghao Zeng and awesome co-authors @nsubramani23 & @TamarRottShaham Details 🧵👇
8
36
9
150
14,443
Ekaterina Lobacheva @ ICML 2026 retweeted
You can now build a robotic scaling law with a cheap robot on 1 GPU. The recipe: 1. Collect 30mins of demonstrations with SO-101 2. Run inference with public RoboJEPA models (a few hours on 1 GPU). 3. Fit the scaling law. details: robojepa.github.io
Introducing RoboJEPA, the first scaling law for multiembodiment world models. We scaled JEPA predictors up to 8B params, showed scaling laws extrapolate and correlated compute scaling with planning performance. We release all checkpoints and the code. robojepa.github.io
5
7
73
3,884
Ekaterina Lobacheva @ ICML 2026 retweeted
New Paper: Subliminal Learning doesn’t need a shared base model 🤯. We show how a “teacher’s” hidden animal preferences reaches the “student” even when both differ in weight initialization, data order, pretraining corpus or even architectures! 🧵
12
28
11
253
23,161
Ekaterina Lobacheva @ ICML 2026 retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
135
796
154
6,569
839,944
Ekaterina Lobacheva @ ICML 2026 retweeted
Quite happy to say that our paper on representation learning for dynamical systems was accepted to #NeurIPS2026 and I'll be presenting it in Sydney this December! Very succinctly, our work, titled "Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems", is about learning linear representations of state for dynamical systems with multiple basins of attraction. Inducing sparsity in a latent representation can be a very powerful method to model multiple basins of attraction with linear dynamics. By training a Koopman autoencoder with a bias for producing sparse latent representations, we find that a sparse-latent model both forecasts the dynamics better and has greater ability to identify the current local regime of the current state compared to a model without sparsity-inducing effects. More broadly, I think our work on sparse Koopman autoencoders could be relevant to both the robotics/world modeling and interpretability communities (just brainstorming here). - Robotics: To my knowledge, switching dynamics are common in robotic systems, i.e. trajectories seem to transition between dynamical modes. With a sparse code, you could get a switch between modes for free. - Interpretability: Sparse autoencoders might not be all the rage anymore, but I think Koopman modeling is still very natural for understanding the dynamics of models that apply the same computation at multiple steps to advance the dynamics. This applies to most recurrent models, such as looped/weight-tied transformers (dm me if you want to discuss!) Thanks to my amazing collaborators @ukreddy126 @mahanfathi @apsarathchandar @RGoroshin. I was thinking about what to say about this (clueless grad student tries to figure out social media) and this post ended up being really late lol, my bad. See you in Sydney! Paper: arxiv.org/abs/2608.29057 Code: github.com/aidanmrli/skae
3
23
3
178
6,949
Ekaterina Lobacheva @ ICML 2026 retweeted
New paper! 🧵 In communication-efficient distributed training, workers take several local steps, and an outer optimizer (often with Nesterov momentum) applies their averaged update to accelerate training. We study when this is stable, and derive an exact criterion (in the mean-square sense).
1
6
4
46
6,559
Join Mila 🔥
Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2027 admission! For more information, visit mila.quebec/en/prospective-s…
1
205
Ekaterina Lobacheva @ ICML 2026 retweeted
How much representational diversity exists around a trained network? A lot! Moving a ViT’s weights by 1.3% while preserving its predictions, its representations end up less similar to the original than a random-init ViT’s ‼️ New paper: Hessian Null Space Continuation (HNC) 💫🧵 (1/12)
9
55
8
356
30,935
Ekaterina Lobacheva @ ICML 2026 retweeted
Can a 1M-parameter model that has never seen an image help an 85M-parameter ViT learn to see? We train a tiny recurrent model on abstract data, then expand its weights. ImageNet-1k top-1: 77.6% → 79.8% in our main setting. Introducing Procedural Core 🧵
7
17
2
108
9,682
Ekaterina Lobacheva @ ICML 2026 retweeted
Introducing RoboJEPA, the first scaling law for multiembodiment world models. We scaled JEPA predictors up to 8B params, showed scaling laws extrapolate and correlated compute scaling with planning performance. We release all checkpoints and the code. robojepa.github.io
38
139
10
808
57,217
Ekaterina Lobacheva @ ICML 2026 retweeted
1/ At COLM? Check out our poster "(How) Learning Rates Regulate Catastrophic Overtraining" during the second poster session on Tuesday in the Grand Ballroom! Not at COLM? Read the thread below 👇
2
8
1
35
4,196
🎓Applying to AI grad programs? Our lab offers 1:1 application feedback for students from underrepresented communities. Apply by Nov 1 👇
🎓 Graduate Program Application Mentorship Applying to grad school doesn't have to be overwhelming! Get personalized 1:1 support from students who’ve successfully navigated the process. 🔗 Apply now: forms.gle/FyjEuLmjcdREF3Ba6 🗓️ Deadline: Nov 1, 2026 👇 Here is what you get:
3
246
Ekaterina Lobacheva @ ICML 2026 retweeted
🎉 Thrilled to share that 8 papers from our lab (and collaborations with lab members) have been accepted at #NeurIPS2026! The work spans RL, world models, dynamical systems, LLM safety & interpretability, tabular foundation models, and forecasting. Huge congrats to everyone 🧵👇
2
20
1
61
4,653
Ekaterina Lobacheva @ ICML 2026 retweeted
meet @mostik_ai! what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can. everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning? we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched. how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance. we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst WIRED has the first external account of the company and the work: wired.com/story/russian-star… full writeup, the setup, and all the numbers: mostik.ai/read-more
297
258
255
2,442
1,283,979
Ekaterina Lobacheva @ ICML 2026 retweeted
Simple ideas often generalize best! Loss smoothing is one of those really simple ideas that works surprisingly well for adaptation under distribution shift. Really excited about this line of work!
Our third invited talk by Sarath Chandar (@apsarathchandar) was about: 🎙️ Stable Adaptation Under Distribution Shift 🎥 youtube.com/watch?v=eddBBYye…
5
57
5,383