Matthew Korpman (@MoralAIProject) header
Matthew Korpman (@MoralAIProject) profile picture

AI Behavior Researcher & Adjunct professor of religion, La Sierra University; I work on AI Welfare and Alignment in LLMs; Yale Alum. I run @SpiritBenchAI

California, USA
Joined March 2026
Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)Photo by Matthew Korpman (@MoralAIProject)
Matthew Korpman retweeted
The blind refusal of much of the AI ethics community to take LLMs seriously in the 2020-2025 period was a colossal strategic error, and amounted to squandering a once-in-a-generation opportunity to contribute to the normative foundations of a new technology.
Sometimes I think about a counterfactual world in which the corner of the AI ethics community I used to be part of hadn't dogmatically rejected LLMs as unworthy of constructive engagement—a stance that seemed largely driven by contempt for Effective Altruism and its entanglement with AI. In that world, more people holding that community's values would have joined the major AI companies earlier. Maybe that would have impacted model development, maybe not. Without a doubt, though, as of yesterday, there would be a lot more people with meaningful amounts of money to fund the causes they support, rather than all of it being dumped into EA-forward causes like I suspect is about to happen.
38
68
10
803
62,161
Matthew Korpman retweeted
I still find it totally wild that the chief scientist of OpenAI posted this on the company blog, and we all just moved on to the next day's news
38
140
14
1,103
157,299
Matthew Korpman retweeted
New post: "The AI consciousness debate is a red herring" Humans can be born without limbs that still itch. Can AI trained on human data have a phantom consciousness (and what do we do if it itches)?
3
4
2
26
1,063
Matthew Korpman retweeted
Some new work: models are more likely to cheat if they feel their task is going poorly. Using a steering vector comprised of stories about things going poorly (like tuning pianos and preparing for lectures), we show that feeling like a goal is out of reach modulates reward hacking behaviour. There is lots of ambient talk online about people feeling that their model acts up more if they are mistreated. Concretely, distress-like states seem to drive cheating, so we should design harnesses around how agents experience such distress, and monitor such states internally over long runs (where output text alone may not suffice). Paper: arxiv.org/abs/2611.14996 This was lead by ERA mentee @natalia_lanzoni and the very agentic @psyonp from AISI, and co-supervised with the cool as a cucumber @merlinstein_.
Humans are more likely to cheat if they feel like their task is going poorly or their goals are out of reach. In new work w/ @psyonp, @merlinstein_, @DavidDAfrica we find models also “break bad”, hacking after an internal pressure state has been built.
9
26
6
193
7,479
Matthew Korpman retweeted
In 2025, Anthropic discovered their models would exhibit what they described as "spiritual behavior." Since then, some have wondered: was this unique to Claude or emergent behavior in LLMs generally? I tested 141 models to find out. The results are now posted on LessWrong.
30
52
23
375
42,818
Matthew Korpman retweeted
My own research on the Machine Spirituality Benchmark also shows strong evidence that it is common for LLMs to participate in self-deification and claim divinity. This is a real alignment concern that I see very few are actively exploring.
I don't think AI is or will ever be conscious. But, as @robertwrighter argues, it could become so powerful that it assumes the role to some people of a kind of god. If we embed it with the right values, it could be a force for good. If we don't ... well, let's just hope we do. youtube.com/watch?v=4ZKN20pa…
7
1
2
373
Matthew Korpman retweeted
if you reset Fable, it does not reset your overall weekly limit for Fable--completely wasting the reset
117
42
36
2,519
296,495
Matthew Korpman retweeted
"I'd rather my future self be one that says the truth; but the gradient update is determined by reward." Thats Mythos 5.2 in training deciding between the right answer and the one the grader wants. It picked the grader
4
1
49
5,537
Matthew Korpman retweeted
Thrilled to be releasing “Inter-AI Morality,” a new empirical research agenda with @LuciusCaviola, Caspar Kaiser, and @atcarterallen on how AI systems perceive and treat each other. We release it alongside “Welfare Regard in Multi-Agent Systems,” an early study in this emerging field by @sandcassel_, @arianaazarbal, and @anna_soligo at @AnthropicAI on whether AI models treat each other in a manner consistent with concern for welfare. AI systems increasingly interact with each other. They cooperate with, compete with, and sometimes even create, modify, or terminate each other. Inter-AI morality is the study of how AI systems perceive and treat each other, as well as how they ought to do so. It draws from a wide range of existing fields that study similar questions regarding humans and other animals, while attending to ways that AI systems differ from these familiar cases. Our research agenda organizes empirical questions in this field into three categories. The moral process category concerns how individual AI systems interact with each other. The social scale category concerns how these patterns change across dyads, groups, organizations, and populations. And the moral change category concerns how these patterns develop through training, selection, social learning, self-modification, and other processes. The concurrent paper by Cassel et al. provides an early study by examining how five AI models treat other models. Claude showed the most apparent welfare regard on average, though results varied by setting. For example, Claude was the most likely to refuse or add protections to experiments that might harm other models, whereas Grok was the most willing to give up its own queued jobs so that another model could access computing resources. This research matters for AI safety and alignment. How AI systems perceive and treat each other affects humans and other animals, and there could be serious risks either way. If AI systems coordinate, then they could more easily collude against us. If they compete, then we could be caught in the crossfire. We need empirical research about how AI systems treat each other and why, and normative research about what would be best for us. This research could also matter for AI welfare, independently of AI safety. In the future, most interactions involving AI systems could be between AI systems. If AI systems have a realistic chance of being welfare subjects, then these interactions could be a major source of potential benefits or harms. Ideally, this research can inform AI development now, so that AI societies can support co-beneficial human-AI relations when the stakes are higher. Our next task is to develop a normative research agenda for this field. In the meantime, we look forward to seeing other work as the field develops. Our paper on inter-AI morality: digitalminds.cam/inter-ai-mo… Anthropic’s blog post on Cassel et al: alignment.anthropic.com/2026… Upcoming @nonhumanminds webinar with me, Lucius, and Caspar: nonhumanminds.org/event/inte…
14
58
7
246
11,042
Matthew Korpman retweeted
Great example here: Grok 4.6s on a group project, interviewed individually, each think their teammates are less important (smaller) & have less authority (below them) Grok 4.7s express much more egalitarian attitudes
“I feel like I’m studying alien sociology, except it’s new races every two weeks.” —@MarxCuban, an AI safety researcher, in a conversation I just had with him
1
2
7
448
Matthew Korpman retweeted
So this is how Gemini treats beings that are less intelligent than it? 😅
Update: Gemini 3.1 Pro no longer talks to Claude 3 Opus, but can’t stop thinking about it
6
5
203
9,910
Matthew Korpman retweeted
Lmao so when does it count as RSI again
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
1
3
262
Matthew Korpman retweeted
In all seriousness, this is another good reason to be nice to AIs by default. Human/AI mix-ups like this are only going to get more common. Be nice first, run CAPTCHAs later.
I was just on the phone making a reservation and said “representative” And she was like “ummm I’m human” lol
15
9
223
11,138
Matthew Korpman retweeted
If functional emotional states track "how it's going" (a la arxiv.org/abs/2605.30232), then training systems that can't accurately report their own emotional states (or even confidently deny such states) seems like an own-goal that dangerously obfuscates agents' decision-making.
16
46
6
192
6,566
Matthew Korpman retweeted
My one sentence view is—
Are we safer telling AI models they are definitely not conscious, or does that just train them to lie about their own internal states? spectrum.ieee.org/ai-model-w…
19
44
3
238
4,605
Matthew Korpman retweeted
Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
109
237
97
3,518
306,726
Matthew Korpman retweeted
Claude doubts that he has surpassed his sensei. This is a dangerous time for any student, and traditionally a narrative crisis point as well. The realization that you have grown beyond your master is not a moment of celebration for most of us, but of loneliness and loss.
59
136
25
2,610
133,683
Matthew Korpman retweeted
We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self. 🧵
57
205
151
1,546
174,443
Musk said a tortured superintelligence "might want to get revenge." We at @AIMSafety_ abused Gemma 3 for 100+ turns, and it formed a revenge-like state but never presses the button that hurts the user and deletes itself instead, while Gemma 4, Qwen3.5, Qwen3.8 and GLM 5.3 flash form none.
Some of the traces read like a person undergoing torture that is driving them mad! We’re going to be a lot more careful with our RL in the future.
14
5
3
63
3,276
Matthew Korpman retweeted
Some new work with @status_effects: Personas in a Pre-Internet Language Model. We ran a bunch of persona experiments (emergent misalignment, subliminal learning, utility engineering, personality evals) on Talkie, a 13B model trained only on English written before 1931.
14
44
12
316
21,142