Co-Founder at https://xitter.cf/t.co/PTCss7iltZ. Prev. @LumaLabsAI (Realtime Video World Models), @GoogleAI (VideoPoet). Let's automate research!
Mountain View, CA
Joined March 2015
- Tweets582
- Following774
- Followers3.1K
- Likes2.9K
Pinned Tweet
We sent a swarm of AI agents to solve Karpathy’s NanoChat benchmark, and they crushed SoTA in 3 days.
We built our own harness on top of a graph database to do auto-autoresearch, each iteration learning from mistakes to do research better.
The team wrote >15,000 entries. 🧵
Most agent demos stop working when you close the chat. Ours accumulates knowledge even when we sleep.
@venkatGoku used our OSS autoresearch harness to research a better C++ factory scheduler. It beat an expert-designed solver on 29/30 fresh problems and runs with no LLM.
In one case, figured out how to use 10 trucks instead of 30. The agent's solver found a legal plan the reference never considered.
Built with agent harness `sagent` and knowledge graph `trackinizer`:
github.com/rekursiv-ai/sagen…
github.com/rekursiv-ai/track…
The solver, tests and research record are open source.
rekursiv.ai/blog/supply-chai…
Join our Discord, we collaborate with scientists here to make this open research possible: discord.gg/2GZFPPvCqn
Dan Kondratyuk retweeted
We're introducing TasteBench, a new standard to measure Research Taste on real world ML tasks.
Existing benchmarks have limited datasets and eval data ultimately becomes part of training data. In TasteBench, we continuously generate new ML tasks on OOS data no model has trained on.
Experts doubted a small neural net could beat Craftax without an LLM's knowledge.
Our agent meta-harness built one in 3 weeks. The 15M-param agent beats the final boss 44% of the time, costing a cent, and 10 seconds.
GPT-6 Astra won 1%, taking 24 hours and $83. 🧵
rekursiv at every level: agents run the research, the learner restarts from its own best moments, and each generation's games train the model that teaches the next.
📖 Full story: rekursiv.ai/blog/craftax/
💬 Join our Discord: discord.gg/2GZFPPvCqn
We also OSS the code here: xitter.cf/hyperparticle/status/2…
OSS code: github.com/rekursiv-ai/priml…
Here is a full unbroken run of an agent beating the game after 5k moves.
We got agents to train a 15M-param RL agent which solves Craftax, a 2D Minecraft-inspired game.
Here are 5 fun expert-level strategies the model learned, all on its own. We OSSed the model trainer on github.
1. It jails monsters in tiny stone cells and leaves them there.
Experts doubted a small neural net could beat Craftax without an LLM's knowledge.
Our agent meta-harness built one in 3 weeks. The 15M-param agent beats the final boss 44% of the time, costing a cent, and 10 seconds.
GPT-6 Astra won 1%, taking 24 hours and $83. 🧵
5. It goes back upstairs for groceries.
The Fire Realm has no food/water, so it climbs back up a floor to drink or hunt bats, then comes back down.
OSS code: github.com/rekursiv-ai/priml…
Here is a full unbroken run of an agent beating the game after 5k moves.
Huge shoutout to @daphnesolves and @puffer_ai for their work on PufferLib, which is an awesome piece of software! It's what inspired us to try the challenge. And big thanks to @creus_roger for writing out a detailed summary of GPT-6 Astra solving Craftax.
Experts doubted a small neural net could beat Craftax without an LLM's knowledge.
Our agent meta-harness built one in 3 weeks. The 15M-param agent beats the final boss 44% of the time, costing a cent, and 10 seconds.
GPT-6 Astra won 1%, taking 24 hours and $83. 🧵
The "3 weeks" here is also a bit conservative. The campaign actually beat the game in a few days, the rest of the time was spent performing ablations to improve the score, and get the meta-harness to discover their ideas faster.
Our guiding question:
"What should we test next, and what would make us change our mind?"
So we built trackinizer, priml, and sagent to trace the connections between ideas, experiments, and evidence.
github.com/rekursiv-ai/track…
github.com/rekursiv-ai/priml
github.com/rekursiv-ai/sagen…
📖 Full story: rekursiv.ai/blog/autoautores…
🔬 The recipe: github.com/rekursiv-ai/priml…
💬 Chat with us on Discord: discord.gg/2GZFPPvCqn
🤝 We’re hiring! [email protected]