PhD student in Oxford (@bold_lab_ai), working on RL & AI Safety 🤖
Joined December 2019
- Tweets32
- Following0
- Followers508
- Likes57
Model-free agents learn to maximise reward without modelling the environment. Right?
In recent work, we challenge this narrative by proving that agents, trained on a sufficiently rich set of goals, encode a unique and accurate world model in their value functions.
1/
Our results soften the boundary between model-free & model-based RL, unlocks hidden generalisation capabilities, and takes a step towards making agents more interpretable & corrigible. But we are just scratching the surface — feel free to reach out with your own ideas!
8/
Work done at @FLAIR_ox and @MATSprogram with Mattie Fellows, @AlexDGoldie, @jonathanrichens, @j_foerst and Oliver Richardson.
🌐 Website: inverting-bellman.github.io
📝 Paper: arxiv.org/pdf/2606.21173
💻 Code: github.com/aletcher/invertin…
⬇️ Agent & implicit WM evolving over training.
We just released LEO (Learning Everything All at Once), a powerful and simple alternative to Hindsight Experience Replay that pushes goals from input to output space in the policy/value networks of goal-conditioned agents. Led by @mitrma 🚀
Alistair Letcher retweeted
Natural evolution's open-endedness leads to beautiful, complex emergent structures and self-organizing behavior 🌱✨. Replicating this in silico is famously hard 💻. Our paper points to a promising direction by evolving populations of competing neural cellular automata with lifelike behavior 🧬🤖 #Isambard
⚠️⚠️flashing lights, rapid cuts, or strobe effects in this thread! 🚨🚨
1/n
Alistair Letcher retweeted
Introducing ✨Infusion✨, our *new paper* made possible by the UK AISI Challenge Fund and Sovereign AI!
1/8🧵 TL;DR
Influence functions are commonly used to attribute model behavior to its training data. In this paper we explored the reverse: whether it's possible to use influence functions to craft training data that induces model behavior?
Huge thank you to my amazing collaborators for making this possible
@LauraRuis @_robertkirk @egrefen @j_foerst and of course
@AISecurityInst and @UKSovereignAI!
Alistair Letcher retweeted
1/ 🪩 Automating the discovery of new algorithms could unlock significant breakthroughs in ML research. But optimising agents for this research has been limited by too few tasks to learn from!
Introducing DiscoGen, a procedural generator of algorithm discovery tasks 🧵
We just released a novel (and hard) benchmark for automated algorithm discovery, led by the excellent @AlexDGoldie ! 🚀
Especially keen on the "automated AI safety" task of model unlearning -- removing hazardous knowledge from an LLM while retaining broad capability. 🙌
We made evolution scale to billion-parameter models. 🤯
Super proud of our team on this wild project!!
Alistair Letcher retweeted
Excited to announce my first paper, with @j_foerst and @FLAIR_Ox, was accepted into @rl_conference 2025!
We establish a new UED method called NCC that obtains strong performance based on principles of optimisation theory.
1/ Excited to share my recent work with IBM Research, "Tight and Efficient Gradient Bounds for Parameterized Quantum Circuits"! We prove stronger, more realistic, and classically computable gradient bounds for variational quantum algorithms (VQAs). Link: quantum-journal.org/papers/q…
5/ Empirical validation:
- We train a qGAN to learn a challenging 2D Gaussian mixture.
- We observe that global contributions to gradients, while initially small, become significant over training. This challenges the notion that only local observables are viable for training.
6/ Huge thanks to my amazing supervisors Stefan Woerner and Christa Zoufal from @IBMResearch Zürich, as well as @MvsCerezo and @qZoeHolmes for their expertise & feedback along the way :)