Incoming PhD @StanfordNLP | BS @PKU1898 | Building continually self-improving AI | Prev @DeepSeek_AI @AIatMeta | DeepSeek-V1/V2/VL/Prover ALE SPIRAL SPICE SPADE
South Korea
Joined February 2022
- Tweets363
- Following614
- Followers1.8K
- Likes6.8K
Pinned Tweet
Continuous self-improvement needs an ever-expanding supply of training environments (goals).
SPADE: one model self-plays the Environment Designer and the Reasoning Agent, writing executable, agentic environments that get harder as it improves. Environment scaling on its own. ♠️
Bo Liu (Benjamin Liu) retweeted
Question to post-training folks: Do we really have to do all that post-training on every base model just to find which ckpt would be the best coding agent? 🤔
What if we can spot potential BEFORE post-training? 🔮 Our method predicts future SWE-bench rankings with ρ=0.964 🔥
⬇️
Bo Liu (Benjamin Liu) retweeted
🧪Can AI agents take a scientific paper and write a meaningful follow-up to it?
We introduce Into the Unknown, an AI agent benchmark of 15 open scientific research problems. Each problem comes from a scientist currently or formerly at US DOE National Labs:
@LosAlamosNatLab @SandiaLabs @BerkeleyLab @AmesNatLab @PPPLab @Livermore_Lab
Bo Liu (Benjamin Liu) retweeted
This new benchmark evaluates nine frontier agents on 15 open scientific follow-up problems developed with domain scientists, producing 135 research papers.
Most of the papers fail integrity checks, including discrepancies between claimed work and supporting execution evidence.
Bo Liu (Benjamin Liu) retweeted
AutoEnvScaling is a joint work with @BaolinPeng, @Benjamin_eecs, @Zhengyan_Shi, @zhaoywang_CS, Hao Cheng, @5000hui, @YaoWenlin, @natashajaques, @shi_weiyan and @JianfengGao0217 :))
@MSFTResearch × @Northeastern × @stanfordnlp × UW × UNC
Paper: arxiv.org/abs/2610.14163
Github: github.com/MSR-Orchard/AutoE…
Page: auto-env-scaling.github.io/
Built on slime (@slime_framework), miles (@radixark) and Harbor (@harborframework)
Bo Liu (Benjamin Liu) retweeted
New impressive work from Microsoft and collaborators.
And it's one of my favorite recent topics: RL environments.
They show that Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 by training on environments it designed itself.
AutoEnvScaling gives environment design to a proposer agent that works in a terminal, looks at the rollouts of the model being trained, and builds new RL environments from them. The trained model's next rollouts guide the next round.
Building environments in a terminal raises the share of valid environments up to 25 times over prompt-only generation, at up to 75% lower cost per valid environment.
With GPT-5.6-sol or Claude Opus 5 as the proposer, Qwen3.5-35B-A3B scores 4.3 and 3.8 points above matched static environment pools. The gain shrinks when the proposer cannot see solver feedback.
The pipeline also produces environments for software engineering, computer use, and GPU kernel work.
arxiv.org/abs/2610.14163
Chat with Paper: academy.dair.ai/papers/autoe…
Bo Liu (Benjamin Liu) retweeted
Our new paper pushes multi-agent recursive self-improvement even further. This time the proposer agent builds whole Harbor sandbox environments to challenge the solver agent. It learns from the solver’s successes and failures to update its own memories, skills, and tools, and build increasingly challenging---yet valid---tasks.
We show powerful benefits both when using a proprietary model as the proposer, and in self-play, when both the solver and proposer are the same open-weight model. In this RSI setting, Qwen3.6-35B-A3B doubles its score on Terminal Bench 4, and performs so well on HiL-Bench it actually exceeds Claude Opus-5.
More evidence of the power of multi-agent co-adaptation to unlock continuous improvement and emergent capabilities.
Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale.
We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.
Bo Liu (Benjamin Liu) retweeted
(2) Can AutoEnvScaling enable recursive self-improvement?
We self-play on Qwen3.6-35B-A3B, since building an environment and solving one are both terminal tasks. Each new checkpoint builds the next round of training environments. Both roles get better: Terminal-Bench 2.1 rises from 41.1 to 53.3, while more of the environments it builds pass validation. The data flywheel closes: the model builds what it trains on and gets better at both building and solving.
a minimal RSI would be: a coding agent writes the repo that trains its successor, the pretrain, mid-train and post-train recipe plus a new harness.
As a follow-up to SPADE, AutoEnvScaling automates one piece of that repo: post-training environments and curriculum for the terminal domain. The environment designer is now a coding agent: it reads the reasoning agent's rollouts, then builds, runs and fixes each environment in a sandbox. Building a terminal task and solving one are both coding, so one model plays both roles and gets better at both. Each checkpoint builds the tasks that train the next. Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 :))
Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale.
We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.
Bo Liu (Benjamin Liu) retweeted
Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale.
We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.
Bo Liu (Benjamin Liu) retweeted
New on our Engineering blog: Creating in-house source control.
In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale.
Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: anthropic.com/engineering/re…
Bo Liu (Benjamin Liu) retweeted
Thanks for commenting! Reef is built to help agents continually improve through real-world usage. If anyone has live agent traffic and wants to collaborate, please let us know (even better if you’re interested in building benchmarks and datasets in this space)! One challenge we face is that current academic benchmarks aren’t sufficient for evaluating continual self-improvement through real-world usage lol.
came across Reef tonight, and the part I like: it keeps agents improving from real usage, not just training runs. it loops serve, observe, grow, commit, and only ships if an update passes eval. a rejected candidate never touches what's live. @ao_qu18465
github.com/Human-Agent-Socie…
Bo Liu (Benjamin Liu) retweeted
My favourite tradition by now:
welcome to the 9th annual @stateofai report!
ai building better ai, world models, physical ai, geopolitics, cyber defense and 12-month predictions.
read it at stateof(dot)ai or watch my editor’s cut here.
a few of this year’s biggest findings follow:
Bo Liu (Benjamin Liu) retweeted
Stanford AI Lab is alive and well at COLM! Check out these SAIL projects this week in SF. 🎉
ai.stanford.edu/blog/colm-20…
Bo Liu (Benjamin Liu) retweeted
Claim: teaching AIs to be better collaborators with humans improves the quality of their work!
🤖🤝Introducing *AutoInteract* 🔁💬🧑💻🧑🤝🧑✨🌱
- creates train data to improve AI's collaborative abilities
- simulates the way humans act & constructs challenging multi-turn interaction tasks
- Improves LiveCodeBench-Pro, Multi-turn LCB-Pro & SWE-together
We expect the recipe generalizes to much more!
Blog post: facebookresearch.github.io/R…
🧵1/5
Bo Liu (Benjamin Liu) retweeted
welcome to the 9th annual @stateofai report!
ai building better ai, world models, physical ai, geopolitics, cyber defense and 12-month predictions.
read it at stateof(dot)ai or watch my editor’s cut here.
a few of this year’s biggest findings follow:
Honored that our SPICE made it into this year's @stateofaireport :)) A self-improving system needs to ground its training signal in interaction with the external world to keep improving. Thanks @nathanbenaich!
welcome to the 9th annual @stateofai report!
ai building better ai, world models, physical ai, geopolitics, cyber defense and 12-month predictions.
read it at stateof(dot)ai or watch my editor’s cut here.
a few of this year’s biggest findings follow:
Bo Liu (Benjamin Liu) retweeted
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Bo Liu (Benjamin Liu) retweeted
(8/8) Guidance-TTT is built on Reef, an inference-native infrastructure for continual, self-improving agents.
If you're building self-improving methods, try implementing them on Reef. We'd love to see what you build 👇
github.com/Human-Agent-Socie…
Bo Liu (Benjamin Liu) retweeted
(1/8) 💡 To make better discoveries, it can be enough to train only the idea generator.
Guidance-TTT trains a small 8B model at test time to propose better ideas, while a frozen model turns them into code. The resulting programs are then evaluated to provide the reward for updating the small model. 🔄
Using only open-weight models and no web access, we outperform the best reported results on 4 discovery tasks. 🧵
Paper and code are out:
📄 Paper: arxiv.org/abs/2610.06269
💻 Code: github.com/Human-Agent-Socie…
