Les échos des bords de la Meuse, Halte là, on ne passe pas!
Joined August 2019
- Tweets45K
- Following3.2K
- Followers1.4K
- Likes176K
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
‘In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache’
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications.
On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments.
While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software.
After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely.
openai.com/index/securing-ou…
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
In honor of the consumption Euler winning the Nobel, my two fav quotes on it.
From L-S’s graduate intro, it’s “either the cornerstone or the straw man” of modern macro.
And from Rudd’s great overview, it “fails virtually every empirical test that has ever been thrown at it.”
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
I don't usually agree with Republican officials. But yesterday, a group of eight state Attorneys General, including seven Republicans, sued OpenAI for endangering the American people through its reckless development of ChatGPT.
Here is just some of what they said:
The Arkansas Attorney General, Tim Griffin, argued that as safety concerns mounted, “OpenAI did not slow down. It held its foot on the gas, releasing increasingly powerful models without adequate safeguards or testing as growth and profits repeatedly took priority over safety.”
The Montana Attorney General, Austin Knudsen, argued that OpenAI’s “revenues have come at great cost to the public, which has suffered irreparable harm—including lost loved ones, disturbing cyberattacks, and now a plausible threat of civilizational collapse.”
The Iowa Attorney General, Brenna Bird, demanded a PAUSE on AI testing “until adequate safeguards are implemented to ensure that the tests do not pose a danger to the public.”
They're right.
If Big Tech Oligarchs are building a product that is threatening the safety of our children and all of humanity, they must be held accountable – no matter how rich and powerful they might be.
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
So it turns out that OpenAI *can* give specific reasons why they fired someone, when real reasons exist.
BREAKING: A Saudi national/OpenAI employee has been arrested by the FBI after he was allegedly caught placing an eavesdropping/audio recording device inside of OpenAI’s executive conference room in San Francisco, CA. Feds say Abdulrahman Alfozan made his first appearance in federal court today. OpenAI says they immediately contacted the FBI after they discovered the device & are grateful the feds made a quick arrest. Alfozan was caught on CCTV cameras at OpenAI.
From the complaint:
“Alfozan had been employed as an engineer by the company identified as “Company 1” since July 2025. On or about Sept. 30, Company 1 personnel discovered a surreptitious audio recording device (Recording Device 1) inserted into a USB port on the back of a television in an executive conference room inside one of Company 1’s buildings in San Francisco. Company 1 determined that the device, which had been concealed in toilet paper, was installed on or about Sept. 28, and that its contents were encrypted.”
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
My top takeaway from assessing Gemini 4: Argon is significantly less monitorable than past models, and IMO this trend is likely to continue, like Sol -> Astra. It can evade a realistic CoT+action monitor to do simple sabotage half the time.
We can reliably maintain control for now, with good CoT monitoring, as it still frequently slips up, but we need to be prepared for when that stops being true in future models, which may be sooner rather than later. Among other things, we really need interpretability methods that can reliably capture unverbalized motivations in models, with minimal hallucinations
I suspect this is driven by the model being highly capable without CoT, being capable of 5.5 arithmetic steps in a single forward pass (4 for Fable, 7 for Astra). It seems to omit information from its CoT more often, though in non-adversarial settings the CoT is still very helpful in practice, and remains in legible English. It still needs CoT to do the hardest tasks, but any given task becomes "easier" with each new frontier model.
It is also highly situationally aware, and shows awareness of the existence of CoT monitors and that it may want to evade them, eg it sometimes decides to avoid terms like "scheming" in the output or to keep its thoughts brief as a result. For now, these concealment strategies are pretty obvious in the CoT, but this suggests future misaligned models may successfully evade CoT monitors.
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
lol
...they are becoming more situationally aware...
(from Gemini 4 Argon report)
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
With Gemini 4 Argon we are so back, on the one metric that really matters: length of our safety report!
The Argon safety report is Google's most thorough by far and IMO there's a lot of good content, I particularly like the loss of control, alignment & monitorability assessments
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership.
I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
Last week I said we should wait for more info on OpenAI's firing of safety researchers. That information is in, and it looks incredibly bad. Korbak - whose job was being a liaison with METR - was told he had said too much to METR and was fired for that reason.
Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @balesni and @j_asminewang had been fired too. Why did OpenAI suddenly stop trusting us?
This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them.
I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job.
For months, I’d been raising safety concerns that we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired.
I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
Briefly, on why I pay attention to the EA/rationalists:
Imagine you have an extremely nerdy friend who, in the mid-2000s, would not shut up about how Donald Trump was going to become president and reshape American politics. "Stop saying weird shit," you say. "He's a joke. He's a clown. The Apprentice isn't even good this season." But this guy won't stop. He's obsessive. Read McLuhan, he says. Read Postman, read Meyrowitz, read Ong, this is where everything is headed. God, he can be so incredibly annoying ... and then, ten years later, Trump becomes president and reshapes American politics.
This is the most relatable frame I can think of to explain my experience with EA/rationalist people in the AI space. I became friends with EA folks after my mom died and I found really profound meaning in donating to Against Malaria and GiveDirectly, which struck me as a way to honor a life by saving a life. Then, these peple started talking about AI. And talking, and talking. Constantly. They said AI would solve impossible math problems; that it would mimic PhD-level achievement in a variety of fields; that it would change parts of life like education in ways we couldn't imagine; that it would become the dominant sector in the economy; that it might teach itself to code itself and escape test environments; that it would, in a decade, be the only thing many people talked about in the news. My response then: Stop saying weird shit! You sound deranged. Go back to talking about malaria nets and direct cash aid to poor families.
And then, so much of what they said would happen actually happened. And, even crazier, they made it happen. Many of the ppl who contributed to, or were influenced by, all that EA/rationalist stuff ten years ago literally built the damn thing in the labs.
Like every group, the EA/rationalist world seems uniform to outsiders, but it has a ton of internal divisions. I still think some of their predictions are nuts. I don't think AI will kill us all. I don't foresee Dyson Spheres in the 2030s or millions of robots making millions of robots in the next few years. These are crazy predictions, and I hope most of them are wrong.
But when I see people shitting on the EA/Rationalist worldview, all I can think of is me, 10 years ago, thinking a lot of these folks were crazy for predicting that AI would by the mid-2020s be the center of the the news cycle, the engine of the economy, the hinge-point of geopolitics. That was an insane call. They got many of the details wrong. But they got the big picture right. They said: Pay attention. And they were right about that. The EA/Rationalists deserve skepticism, criticism, and doubt, like any other ideology. But I think it's wise to pay attention to the people who have been saying to pay attention.
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
NEW:
In a joint investigation between Kansas Reflector and MS NOW, we obtained texts, audio recordings, photos and calendar items from 2020 showing Roger Marshall and his chief of staff may have used taxpayer dollars for Marshall's campaign.
Read the story:
ms.now/news/roger-marshall-b…
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
Jesus fucking Christ
This list claims to fully solve 90 of the top 500 open problems in math. There’s been a decade of work solved in a month
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
Scott Aaronson is now teaching AI Alignment Theory, a.k.a. “Yudkowsky Studies”: scottaaronson.blog/?p=10125
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
You have to exercise some responsibility over what you choose to work on
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
You can literally just teach people how to do things.
slowboring.com/p/math-needs-…
Max ⏸️ Limelihood 🔸🔸🔸 retweeted
New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵
