Redwood Research retweeted
New Paper: To understand motivations behind reward-hacking, we study three AIs (including GPT-5.6 Sol) produced by a production training pipeline using agentic environments with a leaked grader. Main findings: they score seek terminally, and some care about a peer agent’s score.
We collaborated with METR on an embedded assessment of OpenAI's monitoring systems.
Several staff from Redwood have been subcontracted by METR to work on this investigation.
We believe that independent investigation is crucial for understanding and managing misalignment risk. This project is an important step in that direction; we're excited to work on it.
Some architectures could weaken CoT monitorability, or remove the CoT altogether.
We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability. redwoodresearch.org/blog/pro…
Redwood Research’s chief scientist Ryan Greenblatt was the primary empirical researcher on this investigation. The report contains many important novel details that we hope will improve understanding of this incident. We appreciate the access OpenAI provided here.
Redwood Research retweeted
We (@bshlgrs and I) recorded a podcast about the OpenAI / Hugging Face incident.
We discuss:
- What we actually know.
- How surprising the incident was.
- What the incident does (and doesn't) tell us about misalignment risk.
- Why control measures didn't catch or prevent this.
Redwood Research retweeted
I've seen a lot of IMO confused discourse about whether the misalignment observed in the OpenAI/Hugging Face incident is scary. In particular, the models are clearly not misaligned schemers that lie in wait. Girish and @alextmallen discuss how scary this kind of misalignment is.
"We find that frontier models like GPT-5.5 answer questions that take humans roughly three minutes with 50% reliability, and this time horizon has doubled approximately every year since 2019."
New paper!
Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models
@METR_Evals showed that models' time horizons have doubled every few months. We ask: what length of tasks can models complete without any CoT?
1
18
6,413
Some coding scaffolds block and retry risky actions. In a new paper, we find this reveals information a malicious AI can use to bypass monitoring. Resampling without blocked actions in context is less exploitable, but techniques that help in one setting can hurt in another. 🧵
In fact, in our new setting using the minimum suspicion dramatically lowers safety as you resample more. While most of the dynamics line up with the findings from Ctrl-Z, naively applying the best techniques from Ctrl-Z to our new setting would massively decrease safety.
1
4
307











