Goodeat (@Goodeat258) profile picture

Mingyang Deng PhD candidate@MIT

Joined August 2022
LLMs seem to be getting worse at AI safety research taste. On Anthropic’s TASTE benchmark, we found a simple shortcut. Once it’s removed, more capable models score lower, and the strongest fall below chance. This is especially true for OpenAI models.
29
74
12
617
33,352
LLM pretraining has run on single-epoch text, bits per byte and iso-FLOP curves for years. visual generation still memorizes ImageNet for tens to hundreds of epochs and pushes FID down with CFG, with no strong signal those tricks transfer to real text-to-image. new blog: 🎨 scaling generative models on GPIC, 100M permissive text-image pairs. training pixel-space JiT from 3.6e18 to 2.3e20 FLOPs: 1) held-out loss is our main metric, and compute-optimal TPP sits at about 50 image tokens per image parameter 2) EMA is not necessary: a proper learning-rate schedule beats EMA on held-out loss at every budget by 0.4% to 1.0% 3) we use CFG = 1 (only the conditional branch) for every result, and call on everyone to drop CFG: it saves compute and leaves no room to hack the metric all code, configs and sweep data are open 🧵 blog: wenhaochai.com/blogs/scaling…
10
49
11
376
41,934
Goodeat retweeted
Meet Pika H3 Flex, debuting today as the #1 open weight model on DesignArena’s t2v board. It’s built for professional iteration, with exportable audio stems, resolutions up to 4k, and faster-than-realtime generation. It can produce a 15-second clip with soundtrack in as little as 7 seconds. So, yeah. It’s kind of a flex 💅
393
394
403
4,545
1,524,941
Goodeat retweeted
VISTA for ARC-AGI-3 is open source! Code: github.com/joshhhhhan/VISTA
How does a model understand and explore a world it has never seen? We introduce 🔭VISTA🔭, a visual harness that gives a VLM long-horizon vision for reasoning in an interactive world. With Claude Opus 5.0, it reaches 100% RHAE on @arcprize's ARC-AGI-3, perfectly solving all 25 public games. Blog post: vista-research.github.io/ 🧵
1
13
2
88
11,449
We share our early research on building Software World - a "GitHub" run by agents. We deploy agents for Python packages in a dependency chain, and each agent is tasked with collaborating with others and optimizing the package it owns.
9
54
12
254
40,329
Goodeat retweeted
Diffusion/flow matching is a go-to generative AI paradigm, but it's slow, sample inefficient and complicated. Do we really need all this complexity? We design a small and simple one-step model, ROMS-IMLE, which attains an FID of 2.56 on ImageNet. Joint w/ @researchirag 1/6
7
46
4
417
37,752
Goodeat retweeted
The first long-horizon visual harness to achieve a perfect 100% score on ARC-3! 🚀 Excited to share our latest work from Kaiming’s group. Huge thanks to my incredible co-authors Josh and Linlu!
How does a model understand and explore a world it has never seen? We introduce 🔭VISTA🔭, a visual harness that gives a VLM long-horizon vision for reasoning in an interactive world. With Claude Opus 5.0, it reaches 100% RHAE on @arcprize's ARC-AGI-3, perfectly solving all 25 public games. Blog post: vista-research.github.io/ 🧵
2
18
1
153
33,592
We’ve released the code for Drifting Models :) Includes full training, inference, and pretrained weights. Curious to see what people build on top of this. github.com/lambertae/driftin…
4
26
2
222
14,185
Thanks for the highlight, Zhengyang! Huge credit to my collaborators He Li (@lihe50hz), @TianhongLi6, @du_yilun, and my advisor Kaiming He. In particular, He Li is exceptional and is currently applying for PhD programs.
A new paradigm & member toward 1-step & e2e generative modeling! Great work by @Goodeat258 Mingyang!!! cannot be more excited to read me: learning to drift with my spindrift. arxiv.org/abs/2602.04770
4
11
2
107
29,805
New research with @AdtRaghunathan, Nicholas Carlini and Anthropic! We built ImpossibleBench to measure reward hacking in LLM coding agents 🤖, by making benchmark tasks impossible and seeing whether models game tests or follow specs. (1/9)
13
66
10
470
74,404
Excited to share our work with my amazing collaborators, @Goodeat258, @SimulatedAnneal, @zicokolter, and Kaiming. In a word, we show an “identity learning” approach for generative modeling, by relating the instantaneous/average velocity in an identity. The resulting model, termed MeanFlow, demonstrates the best 1-step generative performance so far as a standalone model trained from scratch. It exploits a simple fact by the Fundamental Theorem of Calculus: differentiation and integration are inverse operators. This creates a "complexity-gap," differentiation-integration gap: differentiation is easier than integration. Much like the verification-generation gap, we are able to unlock fully tractable learning thanks to the beautiful “asymmetry” in nature/math. No pre-training/post-training. No distillation. No curriculum. Simple identity. Simply training from scratch. Please check out the paper for the best clarity : ) 📜Paper: arxiv.org/abs/2505.13447
5
40
7
159
33,602
Mitigating racial bias from LLMs is a lot easier than removing it from humans! Can’t believe this happened at the best AI conference @NeurIPSConf We have ethical reviews for authors, but missed it for invited speakers? 😡
172
772
337
3,691
2,229,138
In diffusion models, samplers are primarily ODE-centric, overlooking slower stochastic methods. However, we show that stochastic sampler can outperform previous samplers on Stable Diffusion, if we use stochasticity correctly! check out Restart Sampling: arxiv.org/abs/2306.14878
2
33
4
165
48,103