Changlong Yu (@yucl08) profile picture

Applied Scientist @Amazon | LLM alignment and MLSys | Prev. @HKUST @MSFTResearch

California, USA
Joined July 2026
Changlong Yu retweeted
🧠 How can we keep scaling LLMs with minimal changes in their serving energy? DeepSeek’s Engram and our concurrent work STEM showed the promise of static sparse memory: large embedding tables with efficient access. But can mature pretrained models learn to use a newly added memory within a limited training budget, avoiding costly pretraining from scratch? 🚀 Introducing STAPLE: a recipe to extend pretrained models' abilities with static sparse memory. Serving energy per token stays close to dense and >50% below MoE upcycling in high-throughput regime. Scalable capacity, interpretable memory, and compatibility with MoE upcycling - without any meaningful increase in serving cost. 🧵 Paper: arxiv.org/abs/2610.12599 Code: github.com/Infini-AI-Lab/STA… Website: infini-ai-lab.github.io/STAP…
3
18
5
51
5,813
Changlong Yu retweeted
We observed a similar phenomenon with our own runs: when extrapolated to a horizon of thousands of steps, the early direction found by RELEX overshoots and performanace degrades.
Each week we share ongoing work from across the Base Labs team. We call these releases 'rollouts'. This week we share our progress in understanding the low-dimensional structure of RL training and whether recent discoveries about rank-1 extrapolation of early updates really hold. We find that performance gains largely come from easier problems and fixing doom loops, while harder environments are less linearly extrapolatable.
1
1
8
460
Changlong Yu retweeted
🏅 GPT-OSS-120B + Arbiter reaches gold at IOI 2026 Introducing Arbiter, a code verifier agent. Arbiter + GPT-OSS-120B scores 385.46/600, past the 361.12 gold cutoff. We achieve gold-level performance at <100 generations per subtask, whereas previous search-based systems draw 5k. We're not just scaling TTC, but scaling it efficiently through boosting sequential self-correction. 📝 Project page: horizon-llm.github.io/IOI202… #IOI2026 🧵👇
6
4
6
344
Changlong Yu retweeted
Harness evolution brings a 0.8B model to 100% on ALFWorld. But how much is generalizable improvement vs. overfitting or test-time scaling? Our method, Harness Delta Attribution, analyzes gains and attributes them to these categories. On 4 benchmarks, lots of the gain is overfitting or due to higher compute usage. Future evaluation should study this more! Post: wenwen-d.github.io/blog/harn…
7
30
9
169
33,105
Changlong Yu retweeted
Introducing Khora, our first multiplayer world model. The demo is LIVE. Join an 8-player real-time deathmatch in one shared world now! Built by Ophilus in collaboration with @RhOS_AI , Khora is designed to scale toward Interactive Worlds with unlimited players, with more generated maps and richer gameplay interactions to come. Special thanks to @alibaba_cloud and FITCLOUD for infra and MaaS support, and to @DeemosTech for technical input.
32
35
28
156
774,278
Changlong Yu retweeted
Replying to @Kimi_Moonshot
@Kimi_Moonshot used policy mirror descent (PMD) for RL in Kimi k1.5/k2. Most take it simply as PG+KL with an updating anchor, but this is not the full story. Check our blog for some interesting findings about this algorithm: zhenghaoxu.notion.site/Revis…
1
12
1
38
6,977
Changlong Yu retweeted
On-policy self-distillation is a promising direction for learning from rich textual feedback. But can it really learn from failed trajectories? Our answer: not quite -- unless we let the model actively interpret them. 🧵1/N
10
57
4
476
520,791