Pinned Tweet
Finding a job is a job. I spent months treating mine like one. Spreadsheets, 40 tabs, follow ups I forgot to send. So I did what engineers do with messy processes. I built an app.
Banger guide by Anthropic.
While frontier models have improved at context management, subagents still help in some cases (e.g., parallelizing research tasks with their own context windows).
I still use them, but mostly for complex research work via dynamic workflows and /goal.
For most agent workflows, folding subagents into the main loop via tool calls and skills makes a ton of sense (as Anthropic recommends). That's because newer models are more persistent and better at managing large contexts. Subagents might be getting in the way now, since models can handle much of what subagents were used for. I see a similar pattern with memory (another day).
But you can't just get rid of subagents if you have been using them extensively. You need to run evals (cost, efficiency, etc.) and let that help you determine whether to fold them (and how) into the main agent loop.
The evals should always inform your strategy.
But keep in mind issues that can arise when scaling agent work, such as too many tool calls, context bloat, and unclear separation of concerns. Subagents are still very useful for that.
Lastly, all of this is constantly evolving as frontier models improve. Always keep an open mind and regularly test/eval your workflows and harnesses.
Anthropic is now recommending that companies REMOVE unnecessary subagents from their AI systems.
This is a pretty significant shift in agent architecture, with their team finding that a single agent equipped with skills can frequently outperform complex multi-agent systems, and one customer seeing roughly 2x faster execution.
The emerging rules of thumb:
→ Skills: Teach the agent HOW to do something, including procedures, expertise, and instructions.
→ Tools / MCP: Give the agent the ability to DO things through APIs, actions, and live data.
→ Subagents: Use for parallel work, isolated context, or tasks a single agent struggles with.
The interesting part is progressive disclosure. Instead of spawning specialized agents or stuffing everything into a giant prompt, one agent discovers and loads skills and tools as needed.
Their advice is to fold existing subagents into the main loop, turn their instructions into skills, expose their actions as tools, and run evals to see if performance holds up.
As models have gotten smarter, improving agent architecture is becoming less about what you can add and more about what you can delete.
claude.com/resources/article…
Shane Bennett retweeted
At the top end of models, real-world usability matters a lot more than +/- 2 points on AA. This takes a while to understand for a model, where AA is a sort of potential, but some models hit and some miss (e.g. opus 5 vs 5.5). Seems like 3.1 may be closer to the miss side.
Shane Bennett retweeted
DeepSeek is serving 150T tokens a day on one cluster, more than the open model inference on OpenRouter, Fireworks, Together, and BaseTen combined.
Below is my reconstruction of various inference volume reports across said companies.
Replying to @natolambert
right from the horse's mouth
ah, right. That's "a single cluster", who knows how many they have
github.com/deepseek-ai/DeepI…
Shane Bennett retweeted
By every measure, Opus 4.8 beats human experts as peer reviewers in a study of 124 Harvard social science papers covering a range from statistical analyses to formal models to qualitative work.
As academics, we need to figure out how to adapt the practice of science to AI, fast.
Shane Bennett retweeted
When my first AI book was published in 2024, the best AI could do 4 minutes of human work with a 50% chance of success
When I turned in the draft of Co-Existence earlier this year, it was 17 hours
When it was published two days ago, AI could do 17 weeks open.substack.com/pub/oneuse…
Spent the weekend playtesting my game's BBS mode on my phone. Full TW2002 style terminal, green phosphor, pixel art ships. Type a number to warp. Dialed in like it was 1993 and found a few bugs in my own code within the hour. Fixed them that night and played some more.
Highly recommended read from Google DeepMind and colleagues.
There is a lot of interest in self-evolving agents, but they're hard to get right.
This work highlights how important and difficult memory and retrieval are in agents.
The big question in this paper is whether self-evolving agents keep learning without forgetting.
They tested ReasoningBank, GEPA, and Meta-Harness on four task streams that drift in topic, workflow, or domain, built from BIRD, BFCL, and ScienceWorld.
The weights stay frozen, and the agents adapt by rewriting prompts, memory, and skills.
Self-evolution improves overall scores on most streams. But more adaptation consistently means agents forget more.
Methods that rewrite a shared prompt or workspace make global edits, so they interfere with unseen tasks and overwrite older knowledge when the stream shifts.
Memory retrieval keeps updates more local, but it still pulls in memories from other domains and collides when similar queries need different workflows.
Static benchmarks hide both problems, so test self-evolving agents on a stream that changes over time.
arxiv.org/abs/2610.25858
Chat with Paper: academy.dair.ai/papers/are-s…
Shane Bennett retweeted
We have presents for you! 🎁
- A new WebMCP Skill with best practices for security, development, & consequential flows
- WebMCP Types: TypeScript definitions, with type inference from InputSchema
Give em to your agent, it helps scaffold out WebMCP so fresh and so clean clean
Shane Bennett retweeted
A consequential choice will be made soon by government and companies as they are increasingly flooded with agentic AI inquiries that overcome the frictions that gated access (see the graphs - it has already started!)
Two options: do you make services worse or better as a result?
Shane Bennett retweeted
I’m speaking at the IMF in an hour about the singularity and why I believe it has begun and the choices we'll need to make. Watch live here: youtube.com/live/Rqt9kWoTJbc…
Banger paper from Google DeepMind on long-horizon multi-agent coding.
(bookmark it)
They report that, to the best of their knowledge, this is the first end-to-end autonomous completion of Minecraft by AI agents.
They report that four Gemini 4 Argon agents beat Minecraft end-to-end. They defeated the Ender Dragon in 27 hours and 14 minutes with no human intervention.
Each agent controlled one player through the standard Mineflayer library. No one assigned roles, wrote game logic, or scouted a seed in advance. The agents planned, wrote, and debugged every program themselves, and coordinated through in-game chat and a shared memory.
Over the run, they wrote 2,783 programs and died 605 times. They split the work, shared what they learned, and recovered each other's items after deaths.
If you build agents for long software or enterprise tasks, this is a useful case study. The team had to keep one plan consistent across thousands of dependent steps for more than a day.
arxiv.org/abs/2610.22319
Chat with Paper: academy.dair.ai/papers/a-tea…
Shane Bennett retweeted
router lives in the harness, not a gateway, bc the harness knows the task
blog: langchain.com/blog/how-to-bu…
xitter.cf/neil_xbt/status/210997…
LangChain gave its coding agent three models and a router.
Open SWE, their open-source coding agent, now sends each thread to the cheapest model likely to finish the job.
they tested it on 973 threads, half routed and half always on GPT-6 Astra.
median cost per thread: $0.94 against $2.61. that's 64% lower.
threads ending in a merged PR: 29.2% routed, 27.3% on Astra. LangChain's p-value for that gap is 0.49.
the router sent 56% of threads to the balanced model, 34% to the fast one and 10% to the performance one.
they also tried always using the fast model. that test ended within a day, because engineers reported low output quality.
LangChain's own caveats: the control was the most expensive model, and the router is a proof of concept.
Which three models would you put in the tiers?
I built it because it was my own itch. The best tools come from the builder feeling the pain directly. Using it every day shaped what it became.
It is a simple MVP with a lot of work still to go. But it is my working system and I use it every day. If you are job hunting and want to poke at it: github.com/shanebe-ai/JobOS
Banger paper from Google DeepMind on co-evolving an agent's harness and its weights.
(bookmark it)
If you already log your harness changes and results, those logs can become training data for the model running inside the harness.
This is a nice paper from Google showing how to do this co-evolution.
Specifically, they propose Evo2, a framework for weight-harness co-evolution.
How it works:
One policy plays two roles. As a Proposer, it writes edits to its own harness, and as a Reflector, it diagnoses what each edit changed. Those edits and their outcomes then become training data for both roles, using SFT followed by interleaved GRPO.
The result:
Evo2 reaches 48.3% on SWE-bench Verified and 82.1% on Polyglot, against 41.0% and 62.6% for the strongest baselines. The harness part alone moves a GPT-5-mini agent from 24.4% to 48.3% on SWE-bench Verified.
The evolved harnesses also transfer to other benchmarks without further evolution.
Paper: arxiv.org/abs/2610.13025
Chat with Paper: academy.dair.ai/papers/evo-2…
Recommended NVIDIA paper on memory-efficient post-training RL.
A very interesting technique for scaling RL for LLMs.
The main takeaway is that you can update all of a model's weights with RL and still use less memory than LoRA. At 1.5B, LoGRA averages 7.2 GiB of training memory, while rank-256 LoRA needs 13.2 GiB. Pass@4 is about the same (81.8% and 81.4%), and Pass@1 is about 2 points lower.
If GPU memory is what stops you from running RL on a bigger model, this method is worth a look.
LoGRA never builds a full gradient for the weight matrices it compresses.
As microbatches accumulate, each weight gradient is compressed with a rank-256 random projection that is redrawn every step, and the update is rebuilt from two small matrices. Before each step, it predicts how much the policy will change and scales the step to stay within a KL budget.
Average training memory per GPU drops 45.7% at 7B (31.8 to 17.3 GiB), with peak accuracy close to dense Adam (72.3% and 72.5% Pass@1).
It also trains a 27B model for more than 1,100 steps on a single eight-GPU node, where dense Adam runs out of memory.
Paper: arxiv.org/abs/2610.06647
Chat with Paper: academy.dair.ai/papers/logra…



























