Matthew Lam (@mattlam_) header
Matthew Lam (@mattlam_) profile picture

building @OpenRouter agents always learning agents and AI https://teapawt.org/t.co/OFoB1tETpy @codexreleases @pichangelog @grokreleases

New York, NY
Joined August 2023
Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)Photo by Matthew Lam (@mattlam_)
I will now work towards my goal of being an expert in harness engineering. Harness Eng Learning #1: Codex new compaction Previously codex asked the server to compact the convo at compaction endpoint, that returned an encrypted summary into the future model requests. New compaction instead saves working notes between context windows, starts with fresh context, and retrieve these notes through tool calls when needed. New compaction flow: 1. save state with notes.write_file tool call 2. reset context new_context 3. request for thread_hint to find notes to recover from 4. read from previous notes files, or history tool calls to recover context (notes.read_file, history.read_item) 5. retrieved content enters context as tool results and continue on with new messages Would be interesting to see an eval across long horizon tasks and compactions between the two compaction modes @pvncher . Also, this compaction mode is specifically only for Astra not Sol or others.
13
4
1
69
7,715
Matthew Lam retweeted
Replying to @elonmusk @XFreeze
Could also be titled “waiting for Grok 4.8” 😂 😢
566
229
69
11,127
562,402
Matthew Lam retweeted
As both a user and an inference provider, I have to say @OpenRouter is one of the most customer-obsessed companies I’ve ever worked with. As a user, OpenRouter team consistently provides valuable insights on optimizing token costs through smarter provider selection and routing. As an inference provider, the growth has been staggering. Since onboarding in late September, we scaled from 0 to 100B daily tokens in our first month, then from 100B to 600B daily tokens in less than two weeks. A true rocket ship of a platform. 🚀
“The single biggest change that OpenRouter brings to @PrimeIntellect is velocity.” In under a year, Prime Intellect’s monthly token usage on OpenRouter grew from 11B to 1T+, while cost per token dropped ~60%. They’re betting on a future where no single model wins, and we’re proud to support their growth. Full story: openrouter.ai/customers/prim…
7
13
2
91
6,290
I found the new "You're absolutely right" and it's "I haven't looked at it", especially on Fable 5.2. Fable, what do you think I'm going to ask you next?
2
6
1,701
Go look at it then lol
1
199
The pareto curve of inference providing, GLM 5.3 Flash TTFT down 40%, throughput up, just in the last month!
Here's an animation of cost and speed improving over time since the launch of our new routing algorithm. Counter to what many people believe, this shows how routing actually increases reliability!
3
6
428
MCP has standardized and unlocked so many use cases, excited for interceptors incoming, and more clients to adopt events, apps, and the many others. While building agents we made sure that MCP was a core protocol the harness prioritized
MCP is all about openess and working together. If it is a good idea, we adopt it. Looking forward to working together to standardize it as part of MCP apps and giving it a modelcontextprotocol.io key
3
338
life update, joined @stripe :)
OpenRouter is officially part of @stripe today! More than 10 million developers and companies use OpenRouter, and as they build with more models across more workloads, they need control over what it costs and where their data goes. OpenRouter has built that operating infrastructure to help run AI across every model and provider, with access to new models, uptime through outages, and enforced spend and data policies for each team. Joining Stripe, which has spent years running critical infrastructure for businesses around the world, means we can build it faster and support companies at every scale. Our commitment to neutrality stays the same. Models and providers compete on quality, reliability, and price. You decide what works for your business.
4
13
436
Matthew Lam retweeted
OpenRouter is officially part of Stripe! No single lab will build all the intelligence the world needs. We’re building OpenRouter so every developer and business can choose the best of it, wherever it comes from. Every model stays on equal footing. With Stripe, we can bring that choice to the millions of businesses that run there. Long term, we aim to connect the rails that move the economy’s money with the rails that move its intelligence, and build the best developer experience for both.
OpenRouter is officially part of @stripe today! More than 10 million developers and companies use OpenRouter, and as they build with more models across more workloads, they need control over what it costs and where their data goes. OpenRouter has built that operating infrastructure to help run AI across every model and provider, with access to new models, uptime through outages, and enforced spend and data policies for each team. Joining Stripe, which has spent years running critical infrastructure for businesses around the world, means we can build it faster and support companies at every scale. Our commitment to neutrality stays the same. Models and providers compete on quality, reliability, and price. You decide what works for your business.
74
36
9
626
50,863
love this @bot feature, but it's been working <50% of the time
2
158
Claude steering needs some work, not sure if it's just the desktop or the cli too. Sometimes it'll drop the message, or otherwise it won't answer your steer question and treat it like it's not there continuing it's turn until it replies to both in the end. Codex handles steers better, it should listen to your steered message, determine whether to answer now and whether it should continue it's turn after too.
1
1
4
365
great TLDR breakdown of the Deepseek super agent design article. All harnesses will move towards the separated brain + hands architecture.
this piece is worth the read, but quite dense. here's my (relatively) brief breakdown: super agents are defined by 4 characteristics: - continuous: one agent across threads and access points - productive: responds quickly and completes work efficiently - proactive: responds to events in its environment - persistent: durable execution for high availability and crash recovery these constraints apply to personal agents and agents doing commercial work alike, so the underlying architecture should look similar the primitives behind them: - two level architecture: a router layer keeps interaction continuous, and a worker layer runs tasks concurrently - event driven design: the agent reacts to events in its environment and can schedule runs toward a long term goal - multi channel interaction: context carries across channels, and async execution keeps the agent productive - separate compute, state, and environments: the "separate the brain from the hands" idea, which supports persistence and security and lets the system scale - watchdog: an independent control loop that keeps the whole thing running they're upfront about what's still open: - reliable scale - representing tasks and work without the "session" primitive - agent to agent communication and context standards - security and privacy, as always their core claim is that models need less and less human intervention on individual tasks, so the bottleneck to further productivity gains is shifting to human attention which raises the question i'm most interested in: how do we design human-agent collaboration when human attention is the bottleneck?
2
1
29
1,692
being able to introspect as a team on what to build, where to prioritize spending time, when to ship fast vs not, I think will be incredibly important too. It's so much easier to ship features now, but feels like not many consider whether that is the right product direction as opposed to just getting it out.
being able to introspect your own behaviors, patterns and see clearly how you operate will be one of the most valuable skills for the next year this is the core for automating your work, and freeing up time to focus on more important things
6
352
Matthew Lam retweeted
Hiring exceptional people may be one of the most asymmetric investments a company can make. A great person doesn’t just do their own job well. They make better decisions. Solve harder problems. Raise the standard. Attract other talented people. Help the people around them get better. One exceptional hire can create value far beyond a single role. Investing in great people compounds.
27
63
10
463
19,106
Matthew Lam retweeted
coolest email i have gotten since starting at @OpenRouter: "the compute you sponsored helped identify hits against Bundibugyo ebolavirus in our assays" you check out what @katyenko and her team were able to accomplish here: teapawt.org/katyenko/status/211622…
Binding is nice, but it’s getting boring. If you’re not thinking about biological function, you’re ngmi. At our last @muni_bio hackathon co-hosted with @onepot_ai, we set one ambitious goal: neutralize the latest threat of Ebola using small molecules. The wet lab data is in, and we’re excited to announce that teams were successful, including our agent that repurposed... an anti-itch gel?
 muni.bio/research/hits-from-…
2
5
2
24
4,743
Matthew Lam retweeted
Congratulations to OpenRouter for crossing 10 QUADRILLION annual token run rate today! Token volumes continue to 2x every 10.5 weeks. In the last 9 weeks, OpenRouter processed more tokens than all of time before that.
113
50
56
854
322,646
Matthew Lam retweeted
pretty cool how they did it. It uses the Claude agents sdk, which calls the claude binary, but with the sdk then turn off using claude's features and instead route it back to opencode for stuff like tool use. This is just a dumbed down tldr, but i'd be curious if someone knows how customizable agents sdk let's you go, and if there were any tradeoffs (still needing to use claude features in some areas) with this design, maybe @thdxr ? Will also be cool to see if @pidotdev and other third party harnesses adopt this
you can now use your claude subscription in OpenCode
15
5
93
15,247
Matthew Lam retweeted
very well put, imo the "harness" is essentially the core of your AI product. Modes are very smart now, but the harness still provides the env for the model with tools and access. If your harness is bad, your AI app will still be dumb and can't do much, even if the model is smart.
personally have the exact opposite view here! building a great harness is a deeply core part of building a great product countless examples you can ask an agent to explain from Grok Bot YouTube video, Factory blogs, our blogs at LangChain, OpenAI’s Codex blogs, etc a great harness is a reflection of building a great experience for the user interacting with your agent. it includes - adjusting prompts (which are off part of the harness) - creating bespoke tools/skills that make common workflows a single tool call -> save on context!!! - aggressively examining thresholds for compaction for your given use case - checking if some work can be offloaded to cheaper models in the harness via subagents/routing - building connectors so agents can efficiently access and search over useful context - supporting background work so users get immediate feedback while other work continues to chug along - building memory in a way that matters for your users ^ this is a small portion of the entire world you think about when building a harness for your product Harness Engjneering helps your team build a better product and better understand of what makes your model tick for your use case also every team building frontier vertical products is harness engineering -> it’s absolutely an edge and it’s fine if the new model makes part of the harness unnecessary layer -> there’s so much optimization you can do at every scale across cost, latency, intelligence related note: the importance of having good evals is the ability to quickly hill climb your harness with new models and trust that the config ex: auto research your harness on a good eval set you trust the frontier models can do a lot -> a lot of tasks can just be one-shotted, but many can’t at the intelligence-cost-speed Pareto you need
1
3
8
1,149
Matthew Lam retweeted
recommended reading. had the good fortune to talk with @tianyi a little while ago, and the DSH folks are one of the not so many teams doing innovative stuff in the harness space. the below largely aligns with what we are gravitating towards at earendil as well. single, long lived entity, ad-hoc, freeform, possibly recursive delegation for actual tasks, cross-system event ingestion and triggers. good stuff.
18
45
7
785
107,937
Matthew Lam retweeted
Harness Learning #7: @pidotdev durable vs codex vs opencode code mode there's a crazy amount of detail put into the different code mode implementations, and I just went through a crazy rabbit hole to learn them. Here are a few major things to consider: - codex uses v8 compiling, vs pi durable and opencode use quickJS - pi durable adds wasm on top of quickJS for additional controls like virtual clock, randomness, etc. which allows it to provide recovery and replay for code mode! super cool - plus the wasm allows code mode to spun up in any wasm compatible host - a lot of these harnesses are mostly everything to code mode. There's code only modes where all tool calls are through codemode - EVEN deeper is code mode only + tools are only discovered through code mode search tool! There's also a big difference in how models get the context of how to use code mode and which nested tools are allowed in their codemode script. Pi-durable + opencode more so keep it in the system prompt, whereas codex's code mode catalog will live in its `exec` tool's description. So far I feel there's pros to every design choice, but I do side with @badlogicgames and @thdxr on the quickJS choice especially since a majority of the time is from tool calls itself. Plus I love the pi-durable thought with wasm + durability added in :) I do like codex world state model and how there's separate `exec` and `wait` calls though @ah20im
8
5
2
90
12,053