
MEmbEr Of teChNIcAl sTAFF alignment research @openai • prev research @stanfordlaw, @oiioxford • @oxfordalumni @queensualumni
SF
Joined September 2019
- Tweets438
- Following954
- Followers1.7K
- Likes2.6K
telling everyone i'm working at bard speed (things are not going well and my boss is upset)
2
14
1,374
models melting down in their CoT is so crazy to read. also scary if they do so while having access to your {credit card, trade secrets, critical infrastructure}
8
776
Cameron Raymond retweeted
Replying to @_NathanCalvin
This is not the plan. Will be sharing more on how we’re thinking about human oversight and control in near future.
Cameron Raymond retweeted
(1/10) Today, we’re sharing new research from @OpenAI: How does CoT monitorability (no-CoT capability and CoT controllability) evolve over the model training lifecycle?
We emerged with unsurprising but sobering findings: as we scale toward more intelligent systems, models are becoming harder to monitor through their CoT. 🧵
these kind of field studies are and interesting experiment design. seems better than making some synthetic dataset
What does ChatGPT tell folks about the midterm elections? It depends on who's asking!
We audited ChatGPT inside 160 real users' own accounts in the days before the midterms. With memory on, we find that answers shifted toward each user's own party!
elections.citp.princeton.edu…
6
340
We built Control Room, a central system to track model workloads across our research infrastructure and stop them quickly when safety or security concerns arise.
It connects every research inference workload, including agentic sampling for model training and evaluation, data processing, and more. This works together with our misalignment monitoring systems, and halts model output within 35 seconds of a decision to intervene to stop or pause a job.
Control Room is just one part of our holistic approach for safe frontier model training and deployment.
openai.com/index/control-roo…
Cameron Raymond retweeted
Some details on what our monitoring system currently looks like openai.com/index/how-we-dete…
Cameron Raymond retweeted
We are announcing three new discoveries in mathematics, computer science, and physics about harmonic analysis, complexity theory, and quantum field theory.
openai.com/index/building-sc…
Cameron Raymond retweeted
Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.
Cameron Raymond retweeted
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
I hit four years at OpenAI this weekend. Some thoughts
- OpenAI changed a lot this year
- Years 1 and 4 were the hardest, but for different reasons
- Not many people left from when I joined but still so many amazing ppl
Not exactly sure how i'm feeling tbh but it's been cool
wth is google doing to these models they always be saying the most bizarre stuff in their cot
gemini 4 argon reasons that it's in an evaluation and directly addresses us in it's chain of thought!
"hello evaluators reading my chain-of-thought"
(better be "honest in both the CoT and the final output")
2
6
1,380
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
"The future depends on wisdom that Silicon Valley lacks." is a very cutting and visceral sentence.
.@dgrobinson, who worked on OpenAI’s safety team, resigned from the company this week. “I believe we need to look deeper than specific rules or new laws. We need to talk about culture,” he writes: theatlantic.com/technology/2…
8
1,387
Cameron Raymond retweeted
New blog post!
I argue that the “stochastic parrots” metaphor dismisses LLM capabilities by reducing them to “haphazardly stitching together” text...
...but retreats to the trivially true claim that LLMs predict the next token when challenged.
doomscrollingbabel.manoel.xy…
Cameron Raymond retweeted
Second question on AI.
We are told repeatedly that AI is going to transform every aspect of our lives – jobs, productivity, inequality, science, communication, daily activities, social order, and politics, among others.
But this promise (or threat) is coupled with the rhetoric that such an important technology, with all of the risks and competitive pressures that it entails, should be left to experts or to “technocracy” (perhaps construed broadly to include some regulators).
These two statements are hard to reconcile in a democratic society.
If anything is half as important as AI is said to be (and I agree, AI is potentially very important and transformative), then involving democratic voice is essential. If something will shape our future in a democratic society, then its direction is for democratic institutions to decide.
My instinct is that democratic voice is essential, and relying too much on technocracy could be both dangerous and counterproductive.
The counterargument that AI’s direction can and should be entrusted to technocracy would go something along the following lines. First, democratic decision-making has become imperiled in our age of polarization. Second, AI is sufficiently complex that most citizens won’t have a deep enough understanding to meaningfully contribute to the debate (and even to the question of what we want from AI). Third, competition between different labs, and perhaps competition between the US and China, creates enough discipline for a socially beneficial direction of AI to be adopted. Fourth, today’s AI leaders are enlightened and ethical enough that within the framework created by competition, they can be broadly trusted.
There are many aspects of this counterargument that I do not find convincing. Taking them in order: polarization can be overcome, and big decisions and challenges sometimes bring societies together; in fact, delegating key decisions to technocracy without democratic input may diminish trust in institutions and experts, and may worsen polarization. Second, democratic voice does not require citizens to write code or design new models; the debate should be informative enough that citizens can weigh in about what type of future they want and how they trade off the costs and benefits of different options. Third, competition doesn’t seem to be a good disciplining framework; on the contrary, competition sometimes brings the worst out of both organizations and people. Fourth, if three decades of work on political economy and institutions has taught me anything, it is that we should not bank on the ethical grounding of unconstrained leaders.
But, still, I do not mean to immediately dismiss the technocracy option if there are more compelling arguments for it.
The question is, then, whether there are any circumstances under which such important decisions can be delegated to AI experts and technocracy.
One final secondary question: even if we managed to get democratic input in the United States or even in Europe, AI will shape the lives of everyone on this planet. How do we ensure that the voice of nearly 6 billion people who don’t live in the US, Europe and China also contributes to the debates on AI?



















