This is us in Market Reasoning Models!
We use the market as self improving benchmark to train better reasoning & research models.
New @cbventures market map alert! 🚨
AI agents are moving from talking about markets to trading in them. But as agents get easier to build, where will lasting value accrue?
My new post maps the agentic trading stack through that lens. Here’s where I’m focused:
• Agentic trading harnesses and terminals that turn a user’s goal into an ongoing strategy, with real retention.
• Market reasoning models and data assets that give trading agents a real edge.
• Identity, audit trails, and performance verification infra built for non-human, financial decision makers.
I care less about shiny demos and more about products that get harder to copy as they grow.
If you’re building here, I’d love to hear from you.
Training for RSI with true OOS data and real world ML research tasks
We're introducing TasteBench, a new standard to measure Research Taste on real world ML tasks.
Existing benchmarks have limited datasets and eval data ultimately becomes part of training data. In TasteBench, we continuously generate new ML tasks on OOS data no model has trained on.
4
379
we tried coding harnesses for research and had to reimplement many obvious research frameworks - so we ended up building a harness from scratch
Introducing oh-my-quant (OMQ): a research-first harness that handles all the boring parts so the Agent focus on intelligent tasks
OMQ automatically:
- spawns agent-colleagues to double check work
- sets up independent workspaces per experiment
- monitors & hooks up experiments to any dashboard
- proactively notifies you over telegram on progress
- owns independent reviewers to prevent doom loops
credits to our friends at rekursive AI for sagent and trackinizer! @hyperparticle
there is still a long way to go before LLMs are useful for genuinely hard research tasks
we let GPT autoresearch Deep Learning alphas on crypto perp data and find that GPT:
- doesn't explore parameters well, only minimal changes across iterations
- has poor research taste, proposes experiments like three almost identical learning rates (wtf)
struggles with intelligent evals, ie judging training curves which has no binary criteria
our harness + Opus on MLE bench lite also outperforms other harnesses
MLE bench is the closest public benchmark to our research tasks, and we chose MLE lite because full MLE bench runs just take too long
is the long runtime of ML experiments the reason why there are so few such benchmarks?
our general research harness defines a new Pareto frontier on terminal bench 2.1
improving general research skills seems to help with SWE tasks too
here is how our harness works:
- async orchestration layer
- extremely parallelised execution runtime
- dedicated communication system between main/ subagents
- native research support: efficient tracking, identification & querying of experiments
yesterday we hosted an incredible hack for RSI & autoresearch in SF with Autolab (@ottogin1 and the group are absolute pros at organising hackathons)
we had almost 400 registrations and sadly could not take in everyone, but stay tuned for future hacks.
teaser: there might be one just around the corner next Sunday on #RSI & Harnesses
and the beautiful office was provided to us kindly by fellow YC company usebourne.com - they've just raised $45m to build autonomous white collar workers
3
302














