Arijit Ghosh retweeted
The first priority is to replace FID with something else for me
Replying to @WangYw251
Also noticed this post while I was drafting this thread.
Some interesting conclusions that future research may refer to:
1. Use CFG = 1 because it saves compute and leaves no room to hack the metric.
2. EMA is not necessary.
3. Held-out loss might be a good metric.
However, text-to-image training is noisy, easy to hack, and lacks proper metrics. Curious to see how far this blog will go!
teapawt.org/wenhaocha1/status/2122โฆ
Arijit Ghosh retweeted
๐ What if an image generator could see?
@GoogleDeepMind's Vision Banana showed it can, but kept data, recipe and weights closed. So we rebuilt it fully open:
โจ FLUX.2 Klein 4B/9B finetune w/ public data only
โจ 73.9 vs Vision Bananaโs 69.9 mIoU zero-shot on Cityscapes
โจWe release code + 148 checkpoints for the community/
Aaand we innovate:
๐ We develop a new test-time scaling recipe we call โlegend-ensemblingโ: this gives strong gains by applying more compute without retraining anything.
๐ openvisionbanana.github.io
๐ arxiv.org/abs/2611.15844
A @FunAILab work from A. Jadon, @v_pariza, @FragileGoodwill, @jonaruthardt, @ASophiaKoepke, @abursuc and me.
Arijit Ghosh retweeted
Introducing e-means, an algorithm beating k-means in 10x fewer epochs.
With order of magnitude wall-clock speedups, e-means is released in FAISS and needs no hyperparameter tuning beyond k. It is robust to various scales and datasets. Still skeptical ? Just try e-means [1/10]๐งต
Arijit Ghosh retweeted
Rethinking Neural Thickets Without Weight Perturbations
TL;DR: Reexamines Neural Thickets and shows that perturbing model weights is unnecessary. Simply perturbing output logits, selecting candidates, and voting achieves comparable or better performance, suggesting that output diversity and selection drive the gains.
arxiv.org/abs/2610.40649
Arijit Ghosh retweeted
Neural Thickets (@yule_gan @phillip_isola): add random noise to an LLM's weights, keep the best copies, let them vote. Reasoning jumps, no training.
Why does it work? It's sampling in disguise. Swap noise on billions of weights for one random vector on the logits: same gains ๐งต
Arijit Ghosh retweeted
The Platonic Representation Hypothesis is now a Theorem โผ๏ธ
Train JEPAs separately, on any modalities: they provably learn the same representation up to one rotation.
The result? A 109M JEPA-Text model that aligns with vision models better than 72B LLMs.
w/ @ylecun ๐งต
Arijit Ghosh retweeted
Iโm delighted and honored to share that Iโve been elected an ELLIS Fellow!
A huge thanks to @gabrielpeyre and the ELLIS Unit Paris for the support. Grateful to be part of such an inspiring research community!
๐ข ELLIS expands with ๐ฑ ๐ป๐ฒ๐ ๐จ๐ป๐ถ๐๐ and appoints ๐ญ๐ฒ๐ฌ ๐ป๐ฒ๐ ๐๐๐๐๐ฆ ๐๐ฒ๐น๐น๐ผ๐๐ ๐ฎ๐ป๐ฑ ๐ฆ๐ฐ๐ต๐ผ๐น๐ฎ๐ฟ๐ through the 2025โ2026 selection process
๐ New Units: ellis.eu/news/five-new-unitsโฆ
๐ New Fellows & Scholars: ellis.eu/news/ellis-selects-โฆ
Arijit Ghosh retweeted
Energy-Based Models can now scale better than Diffusion models.
Introducing TEM: Transformer Energy-based Models
TLDR:
โข TEM outscales Diffusion without guidance
โข First scalable contrastive Energy-Based Models
โข Better generalization, test-time scaling, and OOD detection than Diffusion
๐งต
Arijit Ghosh retweeted
Highly recommend! Great opportunity ๐
๐ข PhD position with a flexible start date in 2027. Apply by filling the form if interested: gulvarol.github.io/
Arijit Ghosh retweeted
๐ข PhD position with a flexible start date in 2027. Apply by filling the form if interested: gulvarol.github.io/
Arijit Ghosh retweeted
๐Improve Continuous Diffusion LMs by making them denoise hierarchical token representations!
We introduce Hierarchical Continuous Diffusion LMs (H-CDLMs), which diffuse tokens *and* coarse semantic clusters jointly.
โก๏ธ Better generations for less compute๐งต
Arijit Ghosh retweeted
After MIRA on Rocket League, I was very curious to know what it takes to bring interactive world models to the real world.
It starts with the video autoencoder: how, where and how much to compress. CADRE is our answer, and it's all open ๐
Arijit Ghosh retweeted
Instead of relying on pretrained video latent space we study which tradeoff of spatial and temporal resolution is optimal for real time WM.
We open source all our models and training recipes so you can easily adapt it for your needs!
Arijit Ghosh retweeted
Many world models borrow their video autoencoder from text-to-video generators, and with it, a noticeable lag on every action.
Today we release CADRE, open CAusal Diffusion REpresentation autoencoders that let real-time world models react fast, on a single GPU ๐งต
Arijit Ghosh retweeted
Representation conditioning is better than representation alignment.
The two recent papers reported similar observations. Time for some new trials?
RECON: arxiv.org/html/2610.20608v1
Dino forcing: arxiv.org/abs/2610.11751v1
Arijit Ghosh retweeted
Our latest work: Donโt denoise what you can predict ๐ฆ
Image generators train faster with DINO features. Latent Forcing generates DINO from noise alongside the image, but DINO is fixed by the image. We just predict it and condition on it.
๐ arxiv.org/abs/2610.11751
Hey all, we have a new paper: Dino Forcing Flow Models: Do not denoise what you can predict.
Tldr; Denoising image and predicting (not denoising) related deterministic information (like dino) provides a model with faster convergence!
Read the paper here: arxiv.org/abs/2610.11751v1
Hey all, we have a new paper: Dino Forcing Flow Models: Do not denoise what you can predict.
Tldr; Denoising image and predicting (not denoising) related deterministic information (like dino) provides a model with faster convergence!
Read the paper here: arxiv.org/abs/2610.11751v1
We released our code and instructions for using our pretrained model here: github.com/arijit-hub/dino_fโฆ
1
4
242























