
Research @anthropicai. Previously research @matsprogram, @mila_quebec, @columbia. @join_ef backed founder (acquired).
London
Joined August 2021
- Tweets56
- Following142
- Followers488
- Likes562
Pinned Tweet
🧵New Anthropic Fellows research: We studied mechanisms of "introspective awareness" in LLMs.
LLMs can sometimes detect steering vectors injected into their residual stream. But is this worthy of being called introspection, or attributable to some uninteresting confound?👇
Uzay Macar retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Uzay Macar retweeted
We’re hiring for “psychological design” research at Anthropic. Our aim is to better understand how training impacts a model’s character and alignment. We’re seeking researchers with experience in LLM finetuning, interpretability, and alignment evaluations. More details below.
Happy to have helped a little with this! Models can partly control their own activations when told to, and we don't really understand how. This is an attempt to measure it more systematically. Congrats to Marek* and @joshycodes for driving this and @DavidDAfrica for steering it.
Safely deploying frontier models may require monitoring not only their behaviour, but also their latent space, especially in cases where models are scheming or deceptive.
But in those cases, we must also evaluate how well we can trust these monitors --- a scheming model that could control its activations well could, in theory, confound probes that rely on those same activations.
Some new work on measuring activation controllability in large language models with Marek Kowalski*, @joshycodes, which @uzaymacar and I advised
If introspection is mechanistically grounded, we might eventually query models directly about their internal states (beliefs, goals, uncertainties) as a complement to external interpretability. Our results suggest this isn't unreasonable.
Thanks to my collaborators @Li_Yang_2019 (co-1st-author), @atticuswzf, @PeterWallich, and especially our mentors @Jack_W_Lindsey and @mlpowered.
💻 Code: github.com/safety-research/i…
✍️ Blog: lesswrong.com/posts/BNMLtuDT…
📎 Paper: arxiv.org/abs/2603.21396




