
Ass. prof. of Generative AI. PI of Generative Memory Lab (@DondersInst). Generative diffusion, language diffusion and statistical physics. AI realist.
Nijmegen, Nederland
Joined July 2011
- Tweets12.1K
- Following2.8K
- Followers7.1K
- Likes49.8K
Pinned Tweet
1/?) As promised to Sander Dieleman (@sedielem), we’re finally excited to share:
Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
We show that continuous diffusion can achieve very strong language modeling performance when operating directly on bitstreams, outperforming masked and uniform diffusion baselines, and essentially matching autoregressive models under our evaluation settings.
Luca Ambrogioni retweeted
Can a neural network learn ~2^100 LDPC (Low-Density Parity-Check) codewords from just 10-20 examples?
👉 Grokking in LLMs and memorization-to-generalization transition in diffusion models are related phenomena — the model learns true process of data generation from a small number of training samples.
👉 It is hard to study this phenomenon analytically on natural datasets.
👉 LDPC codeword generation is a precise mathematical process in which the codewords are constructed as solutions to parity check equations.
👉 We train a Dense Associative Memory with a tiny fraction of codewords from the full vocabulary, without giving the network the generation algorithm itself. The model manages to learn the entire vocabulary on its own. It turns out, you only need log(n) samples to learn the entire space of 2^n codewords.
👉 The network only needs approximately P=10-20 training examples to correctly learn the full vocabulary containing ~2^100 codewords.
Our new paper with Andy Lucas: arxiv.org/abs/2610.27384.
Let us know what you think!
Luca Ambrogioni retweeted
Hot take: the community keeps asking whether diffusion LLMs can beat AR. That's the wrong question. Diffusion is a sampling method, and it can sit on top of an AR model to make it faster instead of replacing it.
Speculative decoding already speeds up AR, but it has two long-standing problems. Training a drafter takes a long time, and you have to build a complete pretraining corpus for it. Even then, token acceptance drops sharply on out-of-domain data.
Uno @ssahoo_ offers an elegant way out: it decouples sampling training from capability training. There's no separate drafter. The model itself, plus one LoRA, drafts several tokens in a single forward pass, and the LoRA learns to match the model's own next-token distribution, so there's no drafter corpus to build and no labels to collect.
We applied this to the Looped Transformer Nanbeige4.2-3B: 1 hour of LoRA training gives a lossless 2× speedup on agentic task. We haven't optimized anything yet, and we believe the acceleration can go much further.
The takeaway: diffusion doesn't need to beat AR. Any model that can score its next token can get a fast diffusion mode in an hour.
Page: fengsxy.github.io/LoopUno
Lora Adapters: huggingface.co/liddlefish/Lo…
Luca Ambrogioni retweeted
For the Navier-Stokes proof:
"On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding"
Since I'm now blocked by bluey from replying to this thread, I answer here by quote tweet
the openAI construction has more to do with convex integration than anything else.
In convex integration, the oscillations are typically built by hand and the iteration procedure creates wild weak solutions. The non-uniqueness construction of Albritton–Brué–Colombo used an unstable vortex to split solutions apart. The Córdoba–Martínez-Zoroa singularities amplify vorticity across scales by strain. In the openAI paper, a fluid instability is clever made so that the flow itself generates the oscillations: they grow by drawing energy from the background shear. The Reynolds stress then becomes the force that creates the collapsing smooth vortex. Using a physical instability to create the stress in this way has no precedent, and it is what produces a smooth force rather than a merely rough one.
Some on these ideas are in the literature (e.g. Leibovich-Stewartson 1983) but the way they are combined is definitely insane and definitely nowhere in the literature previously.
I also don't know of anyone suggesting that convex integration would play a role in the creation of a blow-up solution like this.
On a technical level, the paper is unbelievable, maybe beyond what a human could have come up with but not beyond human understanding: there are several different groups of pde experts working on understanding it who say that they more or less understand now. I expect digestions soon
td;lr: anyone saying that this OAI result is unsurprising is bullshitting
sorry!
Luca Ambrogioni retweeted
PhD applications at @GatsbyUCL are open!
Full funding available regardless of nationality.
Includes ✨new✨ Jump Trading studentships in AI with potential application to finance, eligible for employment opportunities with Jump Trading during PhD.
Deadline: 🚨November 9th🚨
Interested in a #PhD in machine learning & theoretical neuroscience?
Applications for 2027 entry to the Gatsby Unit PhD programme now open!
💰 Fully-funded 4-year programme
🤝 Close links to Sainsbury Wellcome Centre & the ELLIS Unit @ UCL (@uclcsml)
ℹ️ ucl.ac.uk/life-sciences/gats…
Luca Ambrogioni retweeted
1/5
Continuous diffusion may be better at discrete reasoning than standard sampling suggests.
We took SRM's MNIST-Sudoku Hard checkpoint (arXiv 2502.21075) and changed ONLY the sampler:
❗ 0.8% → 65.7% valid boards solved, without retraining. Why does standard sampling fail?🧵
Luca Ambrogioni retweeted
The magic of flow map language models is that they capture dependencies between tokens in just 1-2 model passes.
Using this as a drafter for speculative decoding makes a ton of sense, and it's exciting to see this working as well as it does! (Love the shuffle test as well btw!)
Announcing FlashMap: a new speculative decoding algorithm based on flow maps! >6x faster inference on Qwen3-8B 🧵
Project page: jshen.net/flashmap
arXiv: arxiv.org/abs/2610.24365
w/ @Jay41969851 @_ryantlee @jonathanmay @cosmo_shirley
Luca Ambrogioni retweeted
A key difference between continuous and discrete diffusion for language is that the former is (in principle) able to represent uncertainty in intermediate noisy states (i.e. superpositions of possible outcomes).
Binary discrete diffusion is another way to achieve this. Neat!
Luca Ambrogioni retweeted
I like the intuition here: remove strong local correlations to focus on weaker longer-range correlations, which results in better representations to align to.
That same intuition inspired masked-self prediction (arxiv.org/abs/1903.04933) for hierarchical generative modelling.
Luca Ambrogioni retweeted
🎊 Excited to give a tutorial on diffusion LLMs at @NeurIPSConf 2026 with @StefanoErmon @JCJesseLai @mittu1204
📆 Mon, Dec 7, 9 AM AEDT
📍 TikTok Ent Centre (Sydney, Australia)
Topics covered:
> A beginner friendly introduction to diffusion-LLMs
> Designing frontier diffusion-LLMs
> Using Continuous diffusion for LLMs
Luca Ambrogioni retweeted
We’re excited about continuous flows as a scalable foundation for languages, and can’t wait to see what’s next!
Code and checkpoints (0.5B / 7B / 8B) are now available:
📄 arxiv.org/abs/2610.28559
💻 github.com/lafa-gh/large-flo…
🤗 huggingface.co/lafa-hf
Luca Ambrogioni retweeted
Can flow language models scale to billions of parameters and handle real generative tasks?
Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B.
Co-led with @ArnavChavan6 @UdbhavBamba.
Joint work with @wognsfjq96 @hongseu33 @AdtRaghunathan @jw9730
Luca Ambrogioni retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
Luca Ambrogioni retweeted
We’re hiring a summer research intern at the MIT–IBM Computing Research Lab in Boston.
We are looking for a PhD student near graduation.
We’re interested in non-autoregressive architectures for language generation and reasoning, including diffusion models, flows, and EBMs.
Luca Ambrogioni retweeted
🎉 Excited to bring our book The Principles of Diffusion Models to @NeurIPSConf 2026 as a tutorial—this time, all about diffusion LLMs! 🚀
🔹 How do ideas from continuous diffusion extend to language generation?
🔹 What are the latest advances and open frontiers in diffusion LLMs?
From foundational principles to cutting-edge research. Stay tuned!
📆 Mon, Dec 7, 9 AM AEDT
📍 TikTok Ent Centre (Sydney, Australia)
w/ organizers @ssahoo_ @mittu1204 @StefanoErmon
🎊 Excited to give a tutorial on diffusion LLMs at @NeurIPSConf 2026 with @StefanoErmon @JCJesseLai @mittu1204
📆 Mon, Dec 7, 9 AM AEDT
📍 TikTok Ent Centre (Sydney, Australia)
Topics covered:
> A beginner friendly introduction to diffusion-LLMs
> Designing frontier diffusion-LLMs
> Using Continuous diffusion for LLMs
Luca Ambrogioni retweeted
Can a language model be based purely on self-conditioning? 🤔
Yes! Meet LAW 👋 (a.k.a. Loopholing All the Way!)
🧵1/8
Luca Ambrogioni retweeted
🚀 Are you a PhD student pushing the boundaries of AI, robotics, graphics, or accelerated computing?
NVIDIA's Graduate Fellowship offers up to $60K in funding, mentorship from NVIDIA researchers, and a summer research internship!
⏳ Apply by October 30!
blogs.nvidia.com/blog/applic…
Luca Ambrogioni retweeted
THIS IS DISINFORMATION
The PHP exploit Nate is talking about was found by AI-enabled defensive operations. There is no evidence of large scale exploitation. This is a huge win for defense running ahead of offensive.
Luca Ambrogioni retweeted
Here we explore the design space of losses for Strong Stochastic Flow Maps improving FD Dino by 10-50% over the original implementation by using a random ODE approach.
Excited to announce the follow-up to our NeurIPS 2026 Oral: Improved Training of Strong Stochastic Flow Maps!
We improve the training of SSFMs by unlocking derivative-based losses!
Joint work with @antoniofrancaib @rmincam @ChristianKoke @Niklas_TR and @AlexanderTong7
From T. Tao slides:
[Public Lecture]
Math 2.0
Terence Tao
(October 9, 2026)
Slides: teorth.github.io/tao-web/sli…
Recording: aimath.org/videos/terry-tao-…
































