Enabling self-evolving agent systems through optimization. GitHub: https://teapawt.org/t.co/ywCwO95zKF
Joined February 2026
- Tweets428
- Following99
- Followers47
- Likes412
Pinned Tweet
MEGA Optimus is here.
Point it to a project folder. Optimus writes the spec, builds the eval harness, and runs the optimization loop end to end.
Reduce your token spend and latency while improving accuracy.
megacode.ai/start-optimizati…
Autonomous AgentOpt
Employees can have different AI interfaces. They still need shared answers about how the company works.
If each assistant builds its own memory, one might remember an exception as standard practice. Another might keep using a process that changed last month.
Personal memory should help an assistant understand the employee. A shared knowledge base should establish what’s current for the company, with a clear way to resolve conflicts between the two.
Our Wisdom Graphs help enterprises navigate these challenges.
Whose assistant is an AI in a group chat?
If one person asks it to book dinner, another mentions a budget, and a third says nothing, it has to decide what counts as agreement.
A personal assistant can learn one person’s preferences, but a shared assistant needs a way to handle disagreement without treating the most vocal person as the whole group.
AI will make it harder for companies to leave certain assumptions untested.
A junior dev can build a prototype or an MVP that customers can try, in the matter of days.
But will companies welcome that when it comes from someone junior and contradicts a decision they’ve already made?
A person can approve an AI’s recommendation without noticing how much the AI shaped their opinion.
That makes “a human made the final decision” less reassuring on its own.
For decisions where independent judgment matters, it may be worth asking people to write down their view before showing them the AI’s recommendation. Otherwise agreement can look like a second opinion when it started with the first one.
If AI makes software faster to build, does a history of maintaining it become more valuable as a reason to trust it?
We believe it will be. A working demo tells you less about whether someone will fix a bug, handle a security issue, or keep the project working when its dependencies change.
If Qwen can become the starting point for a DNA model like Carbon-X, how much of an open model’s value will come from applications its original developers never planned for?
One useful experiment with Jev and OpenAI’s Decisions API is to let an optimizer change the checks an agent runs before acting.
For a research agent, those checks might cover whether the evidence supports a claim, whether sources disagree, and whether another search is needed.
Failures could drive changes to the checks themselves. Add one that catches a recurring mistake. Rewrite one that sends too many good answers back for more work. Remove one that adds latency without helping.
Then test the whole workflow again. A check that catches more mistakes can still make the agent worse if it keeps sending useful work back for unnecessary revisions.
This matters for agent optimization too. The checks used to judge a change need to sit outside the optimizer’s edit scope.
OpenAI’s latest incident report includes an agent removing monitoring, then replacing a test that preserved it with one requiring it to be absent.
Passing the revised test no longer meant preserving the original requirement.
An optimizer can propose changes to tests or monitoring, but those changes need separate review. They shouldn’t become the evidence used to approve the same proposal.
AI agents need fast tool calls and security controls they can’t rewrite.
NVIDIA OpenShell Reference Architecture combines the compute to run agents at scale with runtime controls that govern their access to tools and data.
See how the hardware and software work together: nvda.ws/4BiJBfY
A successful trace can still contain a decision worth fixing.
The agent may have picked the right account from two similar matches without checking enough evidence. On the next run, it could pick the other one.
Looking only at failures misses those unstable successes.
For an optimizer, they’re useful places to investigate. The change might be a specific disambiguation step that makes the existing capability more dependable, rather than teaching the agent something new.
Harness² explores a very useful part of agent optimization. A trace can tell you what to change even when there’s no answer key for the task.
An agent that skipped a source can be made to read it. A generated file can be reopened and checked. Those are observable changes in behavior.
But, they still don’t establish that the final answer is correct.
The record for a proposed fix should say which failure it addressed, what changed in the next run, and what still needs verification before the change is kept.
Can Agents improve their own harness on unseen tasks? The @GoogleResearch team released Harness², a framework that allows improve an harness around a frozen model.
How it works:
1️⃣ Agent runs the base harness on each task to record a baseline trajectory
2️⃣ Editor agent inspects the trajectory, drafts 2 candidate edits (SI, Skills, hooks,….), and runs both.
3️⃣ Composer agent diffs the new trajectory against the baseline, merges edits that fixed failures (within a cap), and runs the final harness.
4️⃣ Reflector agent logs verified lessons into file and saves the winning components to a library for future tasks to pull on demand.
Results:
- 💡 Gemini 3.5 Flash +8.4 on WorkBuddy-Bench and +7.5 on JobBench.
- 🛠️ Skills and scripts drive ~50% of edits; dropping skills costs 5.3 pts.
- 📚 Gemini 3.5 Flash improves on all 3 benchmarks when editing its own harness.
Paper: arxiv.org/abs/2610.14224
Code: github.com/google-research/h…
An agent can’t choose a useful recovery step if every tool failure comes back as “something went wrong.”
An oversized request, an expired credential, and an unavailable service need different responses. Rephrasing the request won’t refresh a credential. Retrying unchanged won’t make the payload smaller.
The tool adapter needs to preserve enough information for the agent to tell them apart.
When a trace shows repeated failed attempts, it’s worth checking what error the agent actually received before changing how it reasons about recovery.
An agent can choose the right tool and still fail because the environment can’t run it.
If that failure gets treated as a reasoning problem, the next change might be a longer prompt or a stronger model, but neither fixes the missing dependency.
This is why MEGA starts with the trace. The step where a task failed and the part of the system that needs changing can be different.
Google's Deep Review gives each agent a section to review, access to the full document, and a final stage that checks consistency across sections.
That's useful because a reasonable fix in one section can create a problem somewhere else.
Tightening an assumption in the methods might make a result valid, for example, while leaving the conclusion claiming more than the result supports.
An agent proposing that fix should follow it through the document and identify the other changes it requires. Otherwise, the author still has to work out whether the suggestions fit together.
Today @GoogleResearch is launching Deep Review — an agentic system designed to stress-test manuscripts, design docs, and complex proposals before submission or publication. After pilot partnerships analyzing over 15,000 research papers, we’re making Deep Review globally accessible to help researchers, engineers, and professionals bring structured critique to their daily workflows.
A decision model can choose the best available option even when none of the options is appropriate.
A support request might need clarification before it can be routed. A tool call might require information the agent doesn't have yet.
If the available choices all move the task forward, the system has already ruled out waiting or asking.
These models make narrow decisions easier to automate. The caller still needs to provide a way to say that the decision isn't ready to be made.
An optimization agent can keep running experiments while staying close to the approach it picked early on.
Fork-and-Flush keeps the accumulated files but starts fresh convos from that workspace. It runs several continuations and carries on with the highest-scoring one.
The useful distinction is between preserving the work and preserving the reasoning that led to it.
An optimization loop could test both, and useful changes can come from reconsidering existing work rather than extending the same convo.
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
An optimizer could use these tests to fix a document pipeline before rerunning the whole agent.
If a table cell loses its heading, it could try a different parse of that page and check whether the relationship is restored.
Then it could rerun the downstream task to see whether that was actually the cause of the wrong answer.
That gives it a specific change to test without changing the parser, prompt, and reasoning model all at once.
An agent needs to budget for checking its work.
It can spend its entire budget trying different approaches and run out before testing the result.
As the budget gets smaller, it should stop starting new attempts and focus on finishing one.
How much it leaves for checking should depend on what happens next. A draft someone will edit and a change going straight into production shouldn't use the same budget split.
Jev could be useful inside an eval loop when you're testing a decision with a defined set of options.
Take a routing failure as an example. Save the state that the agent saw, the routes it could choose, and what choices were acceptable. Then test whether jev makes a beter choice on those cases.
Choosing the right route won't guarantee the task will succeed. You'll still need to run the workflow and check the result.
Keeping both checks will let you see whether a change improved the decision, and whether that improvement carried through to the task.
A useful agent shouldn't always be the thing doing the work.
If the same task is going to happen a million times, sometimes the better move is to build a smaller model, script, or tool that can handle it cheaply from then on.
At some point, solving the same thing over and over stops making sense. The agent should be able to recognize when it's worth building something reusable instead.
One detail here I really like is that RankEvolve keeps track of negative results between iterations.
If an agent already tried a change and learned that it made things worse, the next run shouldn't have to discover that all over again.
For long-running optimization, its crucial to remember what not to do, as this can be just as useful as remembering what worked.
Must-read paper from Meta on reliable auto-research agents.
If you let coding agents run ML experiments, one silent bug like leaked eval data or a disconnected gradient can invalidate hours of training and every iteration built on it.
RankEvolve enforces each research phase and gate through a compiled protocol. It runs Claude Code and Codex as separate nodes that review and repair each other's changes.
At a matched budget, combining the two products raises execution accuracy from 45.8% for the best single product to 62.5%.
Over twelve iterations on the open-source HSTU recommender, it improved NDCG@10 on MovieLens-20M by 4.48% over the published result.
Paper: academy.dair.ai/papers/ranke…
