@braintrust

Active observability for agents in production.

Joined August 2023
Your team deserves more from your agent observability platform - a single, connected place for instrumentation, investigation, and measurement, enhanced with intelligence. In Braintrust, you can start with an open-ended prompt about agent behavior and carry the investigation into code, evals, automation, and monitoring. New tools like Patterns and Debugger make it easier to find recurring behavior, get to root causes, and act on what you learn. The context you build in one step automatically follows to the next. Work in the platform with an enhanced Loop experience, or move to your coding agent of choice. Read more → braintrustdata.link/agent-ob…
14
11
9
58
43,870
Which AI assistant is the best personal shopper? It depends on how you shop. Dots is best if you care about about price, Muse is the most careful product researcher, and Instinct is the most well-rounded across tasks. Read more → braintrustdata.link/ai-assis…
1
6
491
Braintrust retweeted
We now automatically partition and summarize trajectories when you view them in Braintrust. It's surprisingly cheap with the most affordable LLMs, and makes it much easier to navigate long multi-turn traces.
6
1
34
2,473
Bubble recently launched a completely new version of its core app-building Agent. Braintrust helped make it 1.5x more accurate than its predecessor at visually representing user prompts, while also performing faster and more cost efficiently. Read more → braintrustdata.link/customer…
1
1
4
678
Phrase search with no matches. We tested Nitro against other databases. See the benchmarks → braintrustdata.link/nitro-bl…
3
555
What's new: - Trace your team's coding agents to track costs, usage, and recurring issues - Review agent requests, actions, and responses in a chat conversation style thread view - Prioritize the most important issues in your traces with Patterns severity labels - Send alerts from Loop automations directly to PagerDuty - Ask questions about your dashboards and build new charts with Loop - Trace Cursor agent sessions from the CLI or desktop app Read more → braintrustdata.link/product-…
2
7
518
Run evals at half the cost by using AI provider batch APIs in Braintrust. Large regression suites, model comparisons, and scoring jobs can be batched at a discount if your deadline is flexible. Submit the requests to the provider, collect the results when finished, and track everything in one Braintrust experiment. Now available for both TypeScript and Python → braintrustdata.link/workflow…
1
3
11
2,862
Export your OpenTelemetry traces from the Claude platform directly to Braintrust. Measure production behavior, eval agent quality, and continuously improve performance. Read more → braintrustdata.link/claude-t…
3
2
9
776
Braintrust retweeted
being able to use Codex to automate: text into Muse -> grab results -> save them to Braintrust -> continue until done with all 25 prompts... insane.
We gave AI assistants Muse, Instinct, and Dots the same personal shopping tasks and let them loose on the web. We evaled their product recommendations on response quality, adherence to budget, and how often they recommended the same products. Instinct performed best overall in our eval, but there are tradeoffs depending on how you shop. Dots stood out for finding good options at lower prices, while Muse was best for shoppers with strict guidelines. Read more → braintrustdata.link/ai-assis…
1
1
3
332
We gave AI assistants Muse, Instinct, and Dots the same personal shopping tasks and let them loose on the web. We evaled their product recommendations on response quality, adherence to budget, and how often they recommended the same products. Instinct performed best overall in our eval, but there are tradeoffs depending on how you shop. Dots stood out for finding good options at lower prices, while Muse was best for shoppers with strict guidelines. Read more → braintrustdata.link/ai-assis…
2
9
2
232
19,562
Send your application logs to Braintrust. Connect app logs to agent traces to understand what went wrong and why. Debug with the context from both in one view without switching tools. Learn more → braintrustdata.link/applicat…
3
2
3
21
4,470
Trace, inspect, and improve your coding agent's work. See what's driving costs, find recurring problems across your team's sessions, review suggested fixes, and measure the impact of your changes. Coding agent insights is now in public preview. Read more → braintrustdata.link/coding-a…
2
1
4
12
3,410
Understand where your agent went wrong with the updated thread view in Braintrust. Follow your agent's trajectory as a back and forth chat conversation, with tool calls and sub-agent work in context. Works for complex and custom agents built with any framework. Now available in public preview. Read more → braintrustdata.link/thread-v…
4
1
16
1,204
We ran evals on Opus 5.5 and new GPT-6 models to assess both problem solving and writing quality. - Opus 5.5 had the strongest observed results, but it comes at a cost. - GPT-6 Luna is the standout on value, and is best for running an agent at volume. - GPT-6 Sol balances writing, task performance, and latency, without Opus prices. Read more → braintrustdata.link/opus-55-…
1
1
8
939
What's new: - Get answers about your agents faster with Nitro, Brainstore's new asynchronous query engine - Trace Jev calls from your application or use Jev as a scorer in Braintrust - Send application logs to Braintrust and inspect them alongside your traces - Run Python and TypeScript evals at a discount through provider batch APIs - Group projects into named collections for repeatably assigning access to specific users Read more → braintrustdata.link/whats-ne…
14
18
750
zyBooks builds interactive textbooks. Patterns helps resolve student support requests faster by automatically surfacing issues for their platform engineering team. Try Patterns → braintrustdata.link/patterns…
3
8
502
We evaled the @Resend and @Postmarkapp MCP servers across a range of common tasks to see how agents perform when automating transactional emails. Postmark returned acceptance confirmations faster on average. Resend had higher first-lookup success. Both passed all seven deterministic scorers across the 90 tasks we tested. The choice comes down to your specific workflow, since each provider offers features the other does not. Read more → braintrustdata.link/resend-p…
2
1
2
14
1,230
How does Patterns find failures you haven’t written evals for? Can it catch hidden bugs? How do you know a fix worked? Patterns, Topics, and Loop help you find agent failures, turn them into evals, and track improvements. Read the FAQ → braintrust.dev/blog/patterns…
1
1
4
495
GPT-6.1 Sol promises similar performance to last week's GPT-6 release at one-fifth the token cost of Astra. Does it deliver? We re-ran our eval on how well models can guide users through multi-step problems. GPT-6.1 Sol reached 78.6% problem-solving accuracy at token prices far cheaper than Astra, Opus 5.5, or Fable 5.1. The most notable improvement was how the new model guided users through teaching interactions. Our "teaching quality metric" for GPT-6.1 Sol rose from 64.1% to 77.6%, compared to Astra's 82.2%. The results indicate GPT-6.1 Sol is able to interact with users in the ways they specify, leading to better pedagogy performance. Read more → braintrustdata.link/gpt-6-1-…
3
4
15
806
GetYourGuide is the online marketplace for finding fun activities around the world. Patterns surfaced a failure buried in their logs and made it actionable. Try Patterns → braintrustdata.link/patterns…
12
2
16
2,190
Agents search more often, and faster, than humans. Nitro is how Braintrust keeps up with this query volume. In an A/B test covering more than 1 million queries, including over 10,000 full-text searches, Nitro ran more than 2× faster than previous benchmarks. Read more → braintrustdata.link/nitro-bl…
7
1
15
705