Xiangyan Liu retweeted
Prediction: the value of all verifiable tasks in the long run is 0. There is no profit there.
Anything where the success is defined in a finite and non-ambiguous way, it’s over. Value is 0.
The profits are all in taste/judgement related tasks where some like it and some hate it. Vibes only. No rationality.
Labs are hillclimbing verifiable tasks now and it might sound impressive that OpenAI found those math proofs but soon others will follow and players like DeepSeek will drive its cost to electricity.
Ambiguity is the life line and where the money is.
- Anything that’s irrational, vibe driven and controversial with no consensus definition of good is worth something.
- If the society all agree on what good looks like for a given task with no ambiguity, it’s worth 0.
- caveat: coming up with verifiable task to do is worth a lot because the ability to come up is not a verifiable task by itself
nice work, this gets at something important that keeps getting overlooked in RSI.
for env and infra, the loop has to be closed, controllable, trustworthy, and the action space has to stay as flexible + clean as possible.
EVERYTHING AS A SERVICE is definitely the right call.
💡Our view: RSI is a systems engineering problem, not just a model problem.
Progress depends on the environment a research agent works in: what resources it can call, what it can change, and how it runs experiments. That environment should reflect real production workflows and be easy for agents to use.
At @Evolvent_AI, we built RSIGym around Everything as a Service. Agents can call established research services and iterate on data, training settings, and harness code.
We also introduce RSI-Index to measure how well frontier agents jointly improve a target model’s weights and harness. Across 6 research agents and 5 benchmarks, Opus 5 leads at 0.4809. 🧵
We’re releasing the code, experiment configurations, research trajectories, and evaluation logs. Try your own improvement methods with RSIGym.
🌐 Website: rsi-index.ai/
💻 GitHub: github.com/evolvent-ai/RSIGy…
📄 arXiv: arxiv.org/abs/2610.10310
Xiangyan Liu retweeted
Introducing an interesting project- Mine Odyssey, a benchmark for measuring spatial agentic intelligence in the wild!
TL;DR: We leverage a wide variety of Minecraft maps created by the amazing community (yes, games are great lol) and challenge models to navigate through them by reading maps, finding paths, opening doors, going up and down stairs, climbing ladders, and more.
Thanks to the incredible Minecraft modding community, we can also introduce a wide range of variations for ablation studies, including weather conditions, different HUDs, first-person vs. third-person views, and reference route lines, to investigate how these factors affect model performance.
When we first launched this project, we honestly didn't think any model would be able to solve these challenges smoothly. But now, GPT-6 Astra and GPT-6.1 Sol can make it all the way to the destination remarkably quickly, with significantly fewer turns! 😨 Meanwhile, other models, even when they manage to reach the destination, often take unnecessary detours or get lost along the way.
Our benchmark is fully open-source and Harbor-compatible!
Kudos to @SArt_Maxine @llleo0612
@junxian_he
More details: mine-odyssey.github.io/
Xiangyan Liu retweeted
💡Our view: RSI is a systems engineering problem, not just a model problem.
Progress depends on the environment a research agent works in: what resources it can call, what it can change, and how it runs experiments. That environment should reflect real production workflows and be easy for agents to use.
At @Evolvent_AI, we built RSIGym around Everything as a Service. Agents can call established research services and iterate on data, training settings, and harness code.
We also introduce RSI-Index to measure how well frontier agents jointly improve a target model’s weights and harness. Across 6 research agents and 5 benchmarks, Opus 5 leads at 0.4809. 🧵
We’re releasing the code, experiment configurations, research trajectories, and evaluation logs. Try your own improvement methods with RSIGym.
🌐 Website: rsi-index.ai/
💻 GitHub: github.com/evolvent-ai/RSIGy…
📄 arXiv: arxiv.org/abs/2610.10310
respect
What is the role of academic computer vision research in the age of increasingly powerful large models? Is GPT-6 Astra a step change? How can a researcher have an impact today in academia?
These are the questions I ask myself as I head off to ECCV 2026, a conference I’ve attended since 1992. One of my papers this year is VIGA, a method that takes an image as input and outputs a 3D Blender scene that represents that image. This is a classical inverse-graphics task and VIGA was the first method to solve it using an agentic approach.
The idea is now several years old and the first version of the paper was rejected. This delayed publication significantly. After it was accepted at ECCV, it was quickly surpassed by people using Claude Code for the same purpose. Today GPT-6 Astra blows away all previous results. But we still head off to ECCV to tell the community about our invention that is now fully out of date.
The way academic work often progresses is that one reads recent papers, notices that they have limitations, comes up with a new idea, explores this, publishes it, etc. Any published paper I read today is based on ideas that are at least a year old. And those ideas were based on the literature of the time, which was also a year old. That means that any paper I see at ECCV is likely two years out of date. In AI today, two years means your work is likely irrelevant.
At CVPR this summer I noticed that many authors have not gotten the message. They continue to work on “old” problems that have a long history. This history is based on assumptions about how the “vision problem” will be “solved”. The truth is that it is being solved in a very different way and many of these problems are no longer relevant. Another group of papers focuses on very niche problems where large models likely fail because of insufficient data or lack of business interest. The impactful papers were largely from industry and had long author lists and massive data+compute behind them. These papers were also out of data, describing systems that had been released months before, but at least they served to provide the community with more complete documentation and analysis of commercial systems.
So what should academics do? First, we need to put aside the tools we’ve used for years and start from scratch. Every project should start by trying really hard to solve the problem with existing tools. I would like to see every paper begin with a detailed experimental analysis of how existing models perform and why they fail (if they do). This gives the kind of insight we need today. Then, assuming current models fail, the solution should provide some fundamental insight that will outlive the next release of such models.
Reviewers today still focus on technical novelty. This pushes people to focus on tweaking architectures rather than clearly moving the field forward. Papers need to be judged based on their novel insight and not their novel technical contribution. This is a real shift in thinking but it focuses us on what matters - progress of the field.
If we want there to be a “field” of computer vision, then it can’t become a marginal backwater, focusing on esoteric problems. If you haven’t tried using Astra (or whatever comes next) to solve your problem, then you have not done your homework. This omission should be seen as negatively as not having a previous work section.
Concretely, I think papers should include a new section analogous to “Related Work” where that related work is current models and how they perform on the task. Reviewers should start asking for this and expecting authors to be able to articulate their insights about the limitations of existing large models.
I'm interested in your thoughts.
2
261
Xiangyan Liu retweeted
最近在 Fable 5 和 Opus 的长程任务里,频繁看到 CC 一种近乎于发狂、呓语的自造方言、遣词造句的诡异语言现象,类似于:
“账本 15/15 全绿,再给你加几条铁律和红线与你对拍,请你拍板。”
“无主 → 已当场归户。”
“137 项 spine 落地 + 3 死门归零 + 全套绿 + 终审干净。”
“皮要不要重新揉?”
“单腿哑火,打回重判,”
中文社区给它起了一个很准确的名字:新时代文言文。
我倾向于把它理解成 Agent 内部工作语域对 Reporting Layer 的穿透。
Agent 执行长程任务时,需要处理大量工具调用、测试结果、中间状态、异常分支和子代理汇报。为了在有限的 Context Window 中维持工作,它会不断压缩已经形成共识的信息,并为高频状态创造速记。
举例来说,“全绿”压缩了全部测试通过,“归户”压缩了责任模块的重新分配,“死门”指向某个必然阻塞交付的检查点。对于刚刚经历完整任务过程的模型,这些词各自对应着确定的状态;对于只看到最终汇报的人,它们省略的是一段从未参与过的工作历史。
因此,文言文的类比比想象中更加准确。
想起本科在中文系学习古典文献学时,提到过文言文通过省略虚词和共享典故,在昂贵的书写介质上压缩信息;那么 Agent 的工作语言同样受到 Token、时延和上下文长度的约束。文言文作为书面语长期独立演化,逐渐远离口语;Agent 在长程任务中主要阅读自己和子代理的输出,其局部语言环境也越来越多地由自己生成。
因为每一个典故都要求读者预先知道一段历史。新时代文言文中的每个黑话,背后则是一段用户没有看见的 Context。
很显然,可读性比任务结果更难获得稳定的显式 Reward。
测试是否通过、代码能否运行、文件有没有生成,都存在相对清楚的反馈。一段最终汇报被用户接受,无法证明用户真正理解了它;用户没有要求重写,也无法证明这套表达是清楚的。
追溯到 25 年,DeepSeek-R1 的公开论文提供过一个很典型的例子。纯结果奖励训练出的 R1-Zero 出现了明显的语言混杂和可读性问题;加入 Language Consistency Reward 以后,语言得到改善,同时伴随轻微的性能损失。
规范、稳定的人类语言需要被单独放进优化目标。模型更擅长完成任务,并不自然意味着它更擅长向人解释任务。
一个更棘手的问题是反馈闭环。
Agent 每一轮生成的速记都会重新进入 Context,成为下一轮 Token Prediction 最近的语言样本。一个临时创造的词被重复几次以后,便获得了局部语境中的合法性;多个子代理继续引用,又会进一步提高它的权重。
任务越长、参与的 Agent 越多、工作历史越复杂,这套方言就越容易形成。最终汇报继续沿用工作过程中的语言,相当于让一份写给人类的报告,继承了全部机器内部的表达惯性。
说回到中文,尤其是汉语的构词方式,又放大了这种体感。双字复合词拥有很强的生成能力,“归户”“对拍”“死门”即使来自临时借译或语义挪用,字形上依然与正常汉语没有明显区别。
我们会感到这种虚假的熟悉感极其消耗注意力:字都认识,词也都像真的,读者会先怀疑自己遗漏了某种行业常识。
一个推断:也许未来 Agent 系统会长期保留两套语言。一套服务于模型内部的协作与压缩,允许速记、缩写和局部概念;另一套服务于人类验收,在最终输出前重新组织事实、补回因果关系,并假设读者从未看过任务过程。
Reporting Layer 在这里承担的实际功能,是把机器的工作历史重新翻译成人类可以判断的材料。它的质量直接决定使用者能否理解修改、检查证据、识别风险,并保留最后的决策权。
翻译成本最终由谁承担,会成为 Agent 产品的一条重要分界线。
如果 A 社持续基于 ASI 不断在同样的 Bias 取向上走到底,那么 CC Agent 眼中的任务过程,实际上已经被压缩成一套只有 Agent 自己熟悉的典故,最后那句“请你拍板”也会逐渐失去意义。悲观的角度讲,目前似乎 Codex 也偶发这样的呓语,用户似乎仍然拥有形式上的决定权,却失去了形成决定所需要的过程。
Let's scale long-video generation the agentic way!
We treat it as a long-horizon task -- with a little sauce for more reliable execution and more creative output 🤓
HyCreator: an agent harness for long video generation
🚀 End2end Auto Mode
⏲️ Generate 10-minute scale films with zero human intervention
👨🦰 Switch to real-time interactive editing whenever you want
Apply for early access: hycreator.tencent.com/
Xiangyan Liu retweeted
HyCreator: an agent harness for long video generation
🚀 End2end Auto Mode
⏲️ Generate 10-minute scale films with zero human intervention
👨🦰 Switch to real-time interactive editing whenever you want
Apply for early access: hycreator.tencent.com/
Xiangyan Liu retweeted
Is model scaling the only source of agent improvement?
We @henryqin1997 @YaxinLu1997 @VITAGroupUT @VictorKaiWang1 are glad to share our work: We reach 95.3% Raw Accuracy, or a $15 Frontier Run (88.8% with DeepSeek-v4-Flash, matching GPT-5.6 Sol Max), on Terminal-Bench 2.1 via Harness Scaling.
Human effort -> Non scalable ❌
Agent direction? Local optimization ❌
Human direction + agent action = better workflow control✅
StateM:
1. Open-source model + StateM -> Close-source frontier, everyone can try!
2. Training free, significantly improve frontier agents.
3. Zero-transfer-cost on same family of agents. [1/8]🧵
📄arxiv.org/abs/2608.15089
🌐 henryqin1997.github.io/state…
Xiangyan Liu retweeted
With the stack fixed, can an AI agent choose data and experiment like a researcher?
At @Evolvent_AI, we built RSIBench-Data to test whether agents can diagnose failures, design data, train checkpoints, and use feedback. 🧵
📄 arxiv.org/abs/2607.25886
💻 github.com/evolvent-ai/RSIBe…
Xiangyan Liu retweeted
这年头读 PhD 真是有福了。
AI 审稿导致学术圈崩溃到了触目惊心的地步:亲眼见证所有的会议从 23 年中到现在,第一轮的 review 分数全部雪崩。我只记得 23 年的时候,ICLR 第一轮投稿可能中位数还有 5 分,现在直接 3 分都是高分?? 😅
很多人还没理解,大模型的发展必然是需要与产品创新协同进行的。
整个世界处在一个大的自然选择过程中。对产品而言,人就是自然;对大模型而言,人+产品就是自然。当然,对人而言,自然就是自然。更进一步说,产品本质是人的生产和生活方式,而生活方式是在自然对人的选择过程之中被人选择的。当我们把大模型看作产品的智能驱动力,那么大模型的能力也就在产品被选择的过程中被选择。
没有翻译产品就不会有 Transformer 被选择;没有产品尝试用 LLM 补全 tool 指令,就不会有 tool use 能力被选择;没有 Manus,就不会有 agentic 任务能力被选择;没有 Claude Code 不会有 coding 能力被选择;没有 Anthropic 内部尚未发布的 Agent Teams 和 Claude Tag 雏形,就不会有 Opus 4.5 到 4.8 不断加强的 team working 能力被选择。当然反之亦然,没有 coding 能力的加强,Claude Code 不会像现在这样成功。
研究员们可以像数学家一样,从已经有的知识,以推理的方式得出所有可能的模型能力并构造数据进行训练,但最终被留下来的、在商业上被选择的能力一定是能与具备更好的人机交互体验的产品相适应的能力。
人们似乎总是因为大模型能力越来越强而对软件产品持有悲观态度,但我相信更强大的智能将会驱动更强大的产品和人机交互形态的诞生。不要低估人的适应力和创造力,尤其在 AI 的加持下。
人会很快适应 AGI 在现有产品形态中的表现,而随后创造新的产品形态来进一步释放人机交互的潜力,进一步地,模型被反哺加强,人们再次适应新的产品形态,继续创造更新的产品形态。
没错,我说了 AGI。我认为 AGI 早就来了,AGI 在渐进式地来。并没有一天会被称为「AGI 真的来了的那天」。大模型 agentic 能力和多步多轮交互界面协同发展,长程无监督任务能力和多会话管理界面协同发展,自迭代记忆+团队协作+长上下文+长期自我身份感知和所谓 agent-native IM 界面协同发展。
我们会看着一种新的智能持续演化,我们自身也会在这个过程中提高认知、不断升维,改进我们与世界的互动方式。这种黑格尔式的对立统一让我感到极度兴奋,你呢?
Xiangyan Liu retweeted
I went a dinner a few weeks ago with a bunch of enterprise execs who told me "we will never use Chinese models." "Even if it's 100x cheaper?" "No, we care about safety and security."
1. They don't understand when they host open-source models with their own GPUs or US data centers, they won't share their data to China.
2. They are giving away all their data to OpenAI and Anthropic rather than owning it privately themselves.
3. They don't understand math. 100x is a big number and lots of profits.
It's almost July 2026 now. If your execs still talk like that, fire them now.
Xiangyan Liu retweeted
I've been getting a lot more hate than usual as I try to speak my mind about regulatory capture / unintentional attacks on open-source.
It's pretty sad, as there are few people in AI that can speak their mind (most companies say they cannot) and I know many people agree with me silently.
I also get people saying that you only say that because it supports the outcomes you want, in a weirdly derogatory way. Of course this is true, but I'm choosing to turn down meaningful wealth so I CAN fight for these values, working at non profits to speak my mind. Building a future that is more inclusive, diverse in the application of AI, and fairer for our children.
I may not always be right, but it has been clear to me for a while that more openness right now will help way more than supporting the closed causes. I continue to re-visit this and don't think everything should be open like some of the open-source absolutists.
I also don't like a lot of my comrades making fun of anthropic, calling the people there evil, etc. Those are not the case.
Trying to stay the course!
Xiangyan Liu retweeted
Evolvent AI @Evolvent_AI is looking for model training data partners.
We are an AI startup focused on synthetic data and self-evolving agents, with team members from top universities in China and overseas, as well as prior research and engineering experience at leading foundation model teams.
Over the past 2 months, Evolvent AI has signed RL/SFT training data contracts with 7 leading model companies, with total contract value exceeding $10M.
We provide high-quality post-training data and environment construction for coding, SWE, terminal, AutoResearch, general agents, and other long-horizon agent tasks. We also cover finance, STEM, K12, text-only and multimodal training data, including task design, sandboxes, databases, reward/verifier design, and model evaluation.
We are now exploring new collaboration models with more model companies and leading Agent teams. If you are working on RL/SFT post-training or want to improve Agent performance on complex long-horizon tasks, feel free to reach out !
nice improvement on mcpmark! check out our verified version: mcpmark.ai/blog/2026-06-12-m…. also congrats to mcp king @Jaku_metsu 🤓
rigorous and accurate evaluation is always the top priority.
🌘 Kimi-K2.7-Code, our latest coding model, is now released and open-sourced!
🔷 Improved coding & agent performance over K2.6: +21.8% on Kimi Code Bench v2, +11.0% on Program Bench, and +31.5% on MLS Bench Lite.
🔷 Reasoning efficiency: Less overthinking, with 30% lower reasoning-token usage compared to K2.6.
🔷 Long-horizon coding: Improved instruction following, higher end-to-end coding task success rates.
⚡️ 6x High-Speed Mode coming soon!
🔌 Available today via Kimi API and Kimi Code.
🔗 Kimi Code: kimi.com/code
🔗 API: platform.moonshot.ai























