@MATSprogram

MATS empowers researchers to advance AI alignment, transparency, and security

 Berkeley, CA
matsprogram.org
Joined November 2023
  • Tweets237
  • Following136
  • Followers6.3K
11Media
The most important technology in history is outpacing our ability to understand it. We’re funding exceptional talent to work on it.
7
39
6
481
41,833
MATS Research retweeted
Ryan Kidd
@ryan_kidd44Oct 20
MATS Residency applications close Oct 31! If you're a brilliant computer scientist, physicist, mathematician, governance expert, neuroscientist, lawyer, economist, etc. who wants to work on AI safety & security, we'll empower you to do that work full time. Apply: matsprogram.org/residency?ut…

MATS Residency

MATS research spans AI alignment, safety, interpretability, governance, and model behavior, bringing together diverse approaches to understanding and steering advanced AI systems.

matsprogram.org
MATS Research
@MATSprogramOct 14
The most important technology in history is outpacing our ability to understand it. We’re funding exceptional talent to work on it.
3
9
1
86
5,671
🚨New paper (possibly the most important one I have written yet). We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios. Led by @KamacheeMax.
25
153
37
716
138,109
MATS Research retweeted
Max Kamachee@KamacheeMaxOct 20
It is important to re-state that the threats are not hypothetical, and that we must take action to mitigate the risks of AI-enabled terrorism.
Cas (Stephen Casper)
@StephenLCasperOct 20
🚨New paper (possibly the most important one I have written yet). We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios. Led by @KamacheeMax.
5
5
40
1,545
MATS Research retweeted
Lennart Finke
@LennartFinkeOct 20
New paper out of @MaryPhuong10's @MATSprogram Stream, with @hrdkbhatnagar and @DoddEuodia! We made a testbed for eliciting latent knowledge in a code vulnerability scanning task. See below how we made the sandbagging model organism robust to both prompting and training! ( 1/14 )
4
12
3
121
3,858
MATS Research retweeted
Jake Ward@_jake_wardOct 19
Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
103
234
94
3,480
297,088
MATS Research retweeted
Neel Nanda
@NeelNanda5Oct 15
> The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research. I'm glad MATS is experimenting with new ways to support promising AI Safety research!
MATS Research
@MATSprogramOct 14
The most important technology in history is outpacing our ability to understand it. We’re funding exceptional talent to work on it.
5
14
335
17,401
MATS Research retweeted
Tom Kimpson@KimpsonTomOct 15
Preprint from my work at @MATSprogram is now up: arxiv.org/abs/2610.07476 Looking at technical governance / verification Qs, specifically if power draw can be used to constrain covert compute.

Can Power Draw Constrain Covert Compute? Limits of Analogue...

Frontier AI treaties or agreements on limiting computation require external verification; an external auditor must be able to confirm how much computation actually ran and that parties are...

arxiv.org
1
4
411
You can watch the full conversation on Youtube: youtu.be/SDdt7NpRqjA

How to Move Into AI Safety | MATS Residency

The most important problem in human history needs more brilliant pe...

youtube.com
3
546
The most important problem in human history needs more brilliant people working on it. MATS co-founder and CEO Ryan Kidd explains the talent bottleneck and how the Residency empowers exceptional people to move into AI safety and security. Watch the full conversation.
1
6
65
2,577
The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research. → Up to $285K/yr → Berkeley, London, or DC → Research compute, relocation & visa support Apply by Oct 31 AoE matsprogram.org/residency

MATS Residency

MATS research spans AI alignment, safety, interpretability, governance, and model behavior, bringing together diverse approaches to understanding and steering advanced AI systems.

matsprogram.org
3
364
The MATS Residency is a paid, 6–24 month role for experienced researchers and professionals pursuing independent AI safety research. → Up to $285K/yr → Berkeley, London, or DC → Research compute, relocation & visa support Apply by Oct 31 AoE ↓ matsprogram.org/residency

MATS Residency

MATS research spans AI alignment, safety, interpretability, governance, and model behavior, bringing together diverse approaches to understanding and steering advanced AI systems.

matsprogram.org
1
68
3,609
The best Research Managers know when to step in and where they’ll make the biggest impact. That instinct is rare, and MATS needs more of it to support people doing some of the most consequential AI safety research today. If you want your work to move the needle, talk to us.
1
1
11
1,184
Apply here: matsprogram.org/careers/rese…

Research Manager - MATS Careers

Dec 09, 2025. MATS is hiring: Research Manager.

matsprogram.org
3
471
MATS Research retweeted
Brian@bkida01Oct 6
With support from @MATSprogram and @BlueDotImpact I’m running a mixer event in London for quants who are concerned about AI risks in the evening of 15th Nov. Come meet researchers working in the field (including @NeelNanda5 , MTSs from Anthropic, Apollo Research, and more)! Travel from Europe covered. Apply here by 1st Nov (takes less than 10 minutes) - forms.gle/pL3UjPeWc2yswfM99 Limited space, so earlier applications are prioritised.

AI Safety x Quant Mixer Event

Supported by MATS, sponsored by BlueDot Impact. London AI Safety x Quant mixer event. Provisional: Sunday 15th November, 5-8pm (exact location details will be provided closer to the event). Apply by...

docs.google.com
Brian@bkida01Sep 9
Recently I left quant to work on AI safety. The Huggingface Incident made me extremely concerned that we are not set up to handle the risks. I wrote my this post for people in a similar position to me - why you *can* have impact and how to get started - briandavies.substack.com/p/a…
10
8
3
198
20,231
MATS Research retweeted
Shi Feng
@ihsgnefOct 6
New research led by @PeterNutter1 @DJROYTBURG via @MATSprogram: a better way to update model beliefs that we call grafting. Fine-tune the base model with next-token prediction on synthetic documents (SDF), and simply apply the weight update onto the post-trained model. This works! 1/7
8
19
11
283
19,512
MATS Research retweeted
Joachim Schaeffer
@JSchaeff3rOct 6
Frontier models can now detect AI control interventions. In particular, GPT-6 Astra is incredibly good at this. When we published the initial version of our CIAware benchmark in May most models were not much above chance level, now a step change happened. I think this is a big problem and challenges the effectiveness of control protocols. High control intervention awareness means that interventions leaks information about the monitor which helps circumventing the monitor. Details in the thread 🧵
2
22
8
107
12,577
MATS Research retweeted
Shi Feng
@ihsgnefOct 5
Building Comparative Motivation Profiles with Symmetric Belief Editing will appear at NeurIPS. We refined the methodology and messaging of our performative misalignment work. We are excited to extend this to white-box auditing. by @David00845018 @1700x0 @MinTaywon @JinghuaO via @MATSprogram 1/7
Shi Feng
@ihsgnefMar 18
New post: Sycophancy Towards Researchers Drives Performative Misalignment We found no clear evidence that scheming is more valid than sycophancy to explain alignment faking. 🧵
1
5
27
1,590
MATS Research retweeted
Ben Hagag
@BenHagag20Oct 2
New @NeurIPSConf paper: we're releasing ORBIT: A Framework for Multi-Agent Safety and Security Evaluations. Open source, with configurable attacks & defenses and dynamic topologies, all built on @AISecurityInst's Inspect. When we started this work a year ago, we didn't imagine how urgent it would become by the time we published. Below are some of the interesting results and where we think this should go next. If you want to talk it through, let us know 🧵
3
10
6
65
7,200
MATS Research retweeted
Antoine Vigouroux@teleturnipOct 2
New paper: instead of decomposing a whole neural network, we only dissect the parts used on a task we care about. It's much cheaper, and gives mechanisms you can inspect, erase, or edit to change how the model behaves 🧵
3
31
6
292
24,649
Load more