Open-source infra for self-improving AI
San Francisco Bay Area
Joined June 2024
- Tweets1.6K
- Following171
- Followers407
- Likes1.7K
Future AGI retweeted
One of the easiest ways to misread an AI agent in production is to watch failures without watching usage.
A cluster can grow because the agent is receiving more traffic. Or because it is failing more often. The spike may look the same, but the response should be very different.
That is one reason Error Feed is a personal favorite of mine inside @FutureAGI_ . It shows errors and traffic together, so every increase is not automatically treated as a regression.
When the lines separate, Error Feed keeps the traces behind that change attached to the issue. Engineering can compare a failed run with a working one and check whether the prompt, model, tools, or data changed.
The divergence shows when the problem began. The traces help explain why.
Here is the full investigation workflow:
docs.futureagi.com/docs/erro…
Future AGI retweeted
you can skip my pitch and run the thing.
@FutureAGI_ is open source and self-hostable. the README takes you from the Docker install to sending your first trace. And then you can simulate, evaluate, just basically test, and improve your agent in production with every version release
try it with the agent you already have. i'd much rather hear where it gets awkward than give you another perfectly rehearsed walkthrough.
github.com/future-agi/future…
Future AGI retweeted
How much of its own evaluation should a self-improving agent be allowed to change?
If it can repair the harness, it can fix broken tests - but it can also weaken the checks deciding whether its changes count as improvements. If it can’t touch the harness, it can get stuck trying to satisfy a test that cannot run.
An agent gets a better score after changing its code. That’s one experiment. It gets a better score after changing the grading rules too. Now we have two changes, and we don’t know which one deserves the credit.
In a self-improvement loop, that confusion carries forward. The agent keeps the version that scored better and starts its next attempt from there. If the evaluation has become easier, it can keep accepting worse work while reporting improvement.
The evaluation change might still be correct. It could recognize a valid solution the old grader rejected. Freezing a mistaken grader forever would just reward agents for reproducing its preferences.
But accepting that change creates more work, not less. The unchanged agent needs to be evaluated under the corrected rules too, using the same test cases and execution settings as the new agent. Comparing yesterday’s score under the old grader with today’s score under the new one doesn’t establish an improvement.
The grader change also needs its own tests: examples it should accept and failures it must still reject. Those checks shouldn’t all be rewritten by the same agent in the same attempt. Otherwise, the code, the grader and the evidence approving the grader can all agree while drifting away from the original requirement.
This matters to what we build at @FutureAGI_ . Evaluations give agents feedback on what to improve, but they also provide the evidence for deciding whether to keep a change. Those jobs become tangled when the agent can rewrite the feedback and then use the resulting score to approve itself.
An agent should be able to propose an evaluation change. Accepting that change and accepting the agent’s improvement should remain separate decisions. Sometimes the honest result is “we improved the evaluator; we haven’t shown that the agent improved.” Less exciting than another upward-sloping chart, but a better starting point for the next iteration.
For this week’s eval discussion: what evidence would you require before letting a self-improving agent change the grading rules used to select its next version?
Future AGI retweeted
Vercel's agent feedback + our Error Feed would be a fun combo.
Send a redacted failing trace with the ticket. After the fix, replay the failing run
docs.futureagi.com/docs/erro…
Your agent can now send feedback to Vercel.
Use plain language to explain a bug or feature request, and your agent will use 𝚜𝚞𝚋𝚖𝚒𝚝_𝚏𝚎𝚎𝚍𝚋𝚊𝚌𝚔 via Vercel MCP to attach structured context and submit the comment.
vercel.com/changelog/send-fe…
Future AGI retweeted
Would be fun to try one of those cases with Error Feed at @FutureAGI_
Future AGI retweeted
if an agent books 10pm when you asked for 11pm, why are we asking another LLM to grade it?
Compare the times.
The same for whether a row exists, a payment went through, a file changed.
Use a judge for what those checks can't settle, otherwise that's just == with a token bill.
if code can check it, write the check. save the judge for what code can't see
Future AGI retweeted
Our Error Feed groups repeated agent failures into issues you can investigate, with a suspected cause, suggested fix, and the traces behind it.
The next step shouldn't be copying all of that into a coding agent yourself.
We have MCP tools so another agent can read those findings and pull up the evidence.
One thing we had to fix: the full investigation could be too big for a tool response. So we send the conclusion, fix, and evidence IDs first. The agent fetches the traces it needs next.
Full investigation still in the UI. It doesn't all have to fit in the first tool response.
Check it out: docs.futureagi.com/docs/erro…
Still catching up on last week’s AI agent security incidents? Grab the report and the checklist for your team.
futureagi.com/ai-agent-secur…
Two ways to calibrate an LLM judge with your labeled data, and the difference shows up on the hard cases.
Static few-shot: pick a few labeled examples, put them in the judge prompt, reuse them for everything. Cheap, simple, and it teaches the scoring scale well. It just can't adapt; every input sees the same examples.
Dynamic retrieval: for each input, fetch the closest labeled examples and inject those. More relevant per case, and it scales, since you can keep thousands of labels and still pass only the few that fit instead of bloating the prompt.
Future AGI's ground truth runs the second one. Your labeled set gets embedded; at eval time it retrieves the nearest examples above a threshold you set and hands the judge their scores and reasoning. The cost is real: loose thresholds or noisy labels pull in bad precedents.
If your judge is solid on common cases and shaky on the long tail, the retrieval version is the upgrade. Try it shorturl.at/OhzF2
“Did it call the right tool?” is not tool-calling eval.
It is the first checkpoint.
The real eval starts after that:
- Can it form the right arguments?
- Can it use the returned payload correctly?
- Can it recover from a bad response?
- Can it complete the task across the whole trajectory?
Tool-calling agents don’t fail like chatbots.
They fail like distributed systems with language in the middle.
So we should stop grading them with one flat score.
We wrote a full guide on evaluating tool-calling agents layer by layer.
Full guide -> shorturl.at/gvl3H
Stale judges are easy to miss. If you want to see what your evals catch once the judge can check the live web, try it here: shorturl.at/e3OgG
This week's Friday eval story:
Most teams worry about their agent hallucinating. Fewer check whether their judge is out of date, and the judge usually is. We ran into this while building evals at @FutureAGI_.
One of our internal agents recommended a model that had been released a few weeks earlier. The recommendation was correct and well reasoned. Our judge failed it and flagged it as a hallucination: "This model does not exist."
The judge wasn't careless. It was confidently working from what it learned in training, and nobody had told it the world had moved on.
We laughed about it for a minute. Then we went looking for the other direction, and that was much less funny.
If a stale judge fails correct answers about new things, it can also pass outdated answers about old things. An agent that repeats last quarter's pricing or a retired policy looks perfectly right to a judge that learned the same outdated fact. Both models are wrong in the same way, so the eval agrees with the agent and no one notices.
Engineers building in fast-moving domains will recognize how often this happens. Models, prices, policies, APIs and regulations now change in weeks. Every LLM judge has a training cutoff. The faster your domain moves, the sooner your evals start grading against a world that no longer exists. Nothing crashes or errors, so you never get a signal that it happened.
The mistake is treating two different questions as one:
-- Is this answer consistent with what the judge knows?
-- Is this answer true today?
For a lot of evals, those used to be the same question. For agents working with live information, they aren't anymore.
That changed how we build Evaluators. Our eval agents can check the internet and the customer's own knowledge base before scoring, so they verify against current information instead of relying on memory. When a judge can't verify a claim, it says so. "Couldn't verify" is a different result from "wrong," and treating them as the same is how good answers get failed and stale ones get passed.
The broader point is that a judge needs the same access to current information as the agent it's grading. If it can't see what the agent can see, it isn't grading the answer. It's comparing the answer to old training data
What's the fastest-changing fact your agent depends on, and when did your judge last learn it?
Four evals at 95 percent each can still mean only 80 percent of your outputs are good.
Each eval passes on its own most of the time, but they fail on different outputs, so fewer pass all of them together. That all-pass rate is the real quality of your agent, and it is always lower than the individual scores that make things look fine.
You can't see that number while the evals sit in separate columns. A composite eval gives it to you: it combines the checks you already run into one result per output, passes only if every check passes, and tells you which check failed.
Small change: one combined result instead of four. Big change: you finally see how good your agent actually is, not how good each check looks on its own.
Try now- - shorturl.at/BzWy0
Voice evals are only useful if they can read the full call - not just the transcript.
Future AGI now maps recordings, latency, interruptions, outcomes, summaries, and runtime metrics more reliably across every run.
Less setup debugging. More clarity on what happened.
Run a voice eval →app.futureagi.com/
A customer got refunded twice.
Let's read the agent's trace backwards, one step at a time, until we find out why.
So the fix belongs in the payment tool, which has no way to tell a retry from a new refund. A better prompt wouldn't have touched it.
Error Feed groups every trace that failed this way and points at the layer the fix belongs in, which here is the tool.
Reply with link: docs.futureagi.com/docs/erro…
Customer support is probably the best place to learn what a voice agent can actually do.
The calls are repetitive enough to find patterns, but messy enough to expose the real problems: interruptions, slow responses, unclear intent, and awkward handoffs.
That’s why the first workflow should be narrow and measurable. A good idea is to start with order status, account questions, appointment changes, or another task where success is easy to verify.
Test it before launch. Watch how it performs with real-world variation. Then widen the scope one workflow at a time.
We wrote a practical playbook for taking voice agents from that first workflow to reliable customer support at scale. shorturl.at/HFePW
