RL agent interpretability at @ARTAIBath. Member at @GivingWhatWeCan. Rank 3 HCIM in @Runescape. Dad. Past: Co-organizer of @minerl_official. Quant.

UK
Joined July 2017
Breaking: @OpenAI’s Minecraft agent VPT kills villagers! In other news: our paper on dissecting VPT with various interpretability techniques was accepted at the Mechanistic Interpretability workshop at ICML. 1/7
6
19
7
118
33,067
Cool work! I've worked on AI+Minecraft for 8 years and do a bit of Minecraft speedrunning. I've read the paper. Here's my review with assorted thoughts, clarifications, potential mistakes, questions and ideas. Overall, I like it. Outsource the low-level control to Mineflayer, and have generalist coding agents figure out the high-level strategy, which worked on the very first seed! I also respect the transparency, clearly stated limitations and not overhyping the results **cough Dreamer cough Nature**. Good job. CC @maxencefaldor @robertarail Thread below.
Four Gemini 4 Argon agents beat Minecraft from scratch in 27 hours. Operating in a sandbox using only code and text, they wrote 2,783 programs, divided up jobs over chat, shared notes, and debugged errors together - all with no human help ↓ xitter.cf/maxencefaldor/status/2…
1
4
84
Potential mistakes/clarifications: "and the random-seed speedrun record is under seven minutes" that's a 1.16.1 time. 1.16.2+ speedrun fastest time is 13-14 min, Gemini used 1.20.4, so that's a more relevant comparison. "collecting blaze rods and ender pearls in the Nether" - you can collect ender pearls in the overworld too, for example as drops from Endermen, trading with villagers, or from chests in the stronghold. "and an ender pearl, dropped by Endermen" - can also be obtained by trading with piglins in the nether or villagers in the overworld, both valid strategies used by speedrunners. "... requires ... destroying the ten crystals that heal the dragon in the End" - you don't need to destroy the crystals, speedrunners generally don't do that, just wait for a perch and kill it with bed explosions, pretty easy tbh (36:22 here: youtu.be/c2Y6duJD_jI?t=2182) "The estimate was 43m from the actual portal room." - ender eye in 1.20.4 points towards the corner of the 16x16 chunk with the entrance staircase of the stronghold (that leads nowhere if you go up). So Gemini probably estimated this coordinate very precisely, but then had to find the portal room from that starting position. Iron tools are not required, speedrunners get iron pickaxes to be able to mine gold blocks in bastions for trading. Unless you count the iron bucket as a tool. "DreamerV3 collected diamonds from scratch with reinforcement learning" - not in Minecraft, but in an extremely simplified version of Minecraft, easier even than Crafter. Crafter is a 2D toy version of Minecraft created by the same author... (tweet) "requires ... using math to locate a hidden underground fortress" - it does not require math or even rough triangulation, you can just follow the ender eye where it leads you and you'll reach the stronghold.
1
2
32
Thank you for the work, it was a very interesting read. Looking forward to what you do next! Hopefully it will produce as many creeper holes as the bottom of this screenshot. Is that the spawn? 😆
2
15
Specification gaming...? me: "don't put snot on my face!" 3yo: "I put snot on your neck!"
2
50
Not too surprising, but still pretty cool! Good job! Esp since it used the latest MC version. VPT used 1.16 (latest at the time), which coincidentally is also what most MC speedrunners use nowadays because the new versions make speedrunning harder.
I asked GPT-6 Astra to mine a diamond in Minecraft using computer use, then went to sleep. Woke up to a diamond in its inventory 🤯🤯🤯 Setup in the replies.
4
399
Saw this and my gut reaction was: "OMG, The rogue AI hacker-agent swarm strike begins!". I do have multiple 2FAs. In reality, it's probably because I had an insecure 2FA (SMS, now removed). I'm not at OA, I collaborated 4y+ ago. But... Could I have become an attack vector?
2
77
When you're trying to be nice to your AI... "I've set a timer for one minute with the label 'please'"
1
86
First Anki card for the 3yo. We took the picture ourselves!
3
227
Got a spotlight at the ICML mechinterp workshop tomorrow! MultiSTEVE-1s - a model zoo and interp suite of murderous Minecraft derps, ahem, agents. Come say hi at poster spot #317 in Hall C at 11:00. Spotlight talks at 10:30.
1
2
5
303
Paper: openreview.net/pdf?id=lY7miv… Credit to @Wurst_Imperium for the awesome meme pic! It's on the poster, on the spotlight slide, and on my t-shirt :)
1
2
120
We've seen @OpenAI VPT attack villagers thinking they're trees. Now we have @GoogleDeepMind SIMA 2 thinking forge bellows are "a dark spiky creature", what's next? :) @jeffclune have you noticed this one?
1
477
We are looking to appoint a Research Associate in Reinforcement Learning. Closing date: 18 Sep 2025. bath.ac.uk/jobs/Vacancy.aspx…
2
349
I've been working on a @3blue1brown style video explaining our research on attention patterns of OpenAI's Minecraft agent VPT. 1min teaser below. You can see a longer WIP version in my recent talk at @cohere (from 24:55): youtube.com/watch?v=p_vwKfhX…
1
2
1
13
1,569
Come and have a look! In the 2nd half of the talk I'll be showing the first public sneak peek of a 3Blue1Brown style video on VPT interpretability I've been working on.
Be sure to check out our guest speaker session with Karolis Jucys next week on Tuesday, May 27th as he presents "Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agent." Thanks to @rahul_narava and @gustiwinata_ for organizing this session!
4
291
One day, we will achieve this true feat of intelligence. Surprised they haven't hard-coded it by now.
2
183
The true test of intelligence. Gemini 2.0 still fails it. I wonder if o3 will pass.
2
2
368