Robert Claypool (@RobertClaypool) profile picture

safety engineer, decision loop engineer, AI operational alignment architect

Nashvile, TN
Joined March 2009
our safety cannot depend on successfully containing machine intelligence because it's not going to happen
OpenAI red-teamed their hardened containment infrastructure by asking teams of frontier-model agents to try to break through its defenses. They easily escaped in novel ways. Quoting: 'Our agents VM escapes showed why containment must extend beyond the sandbox. To test the surrounding perimeter, we gave teams of agents control of a host machine and asked whether they could extend that compromise to reach the live internet. After the Hugging Face incident, we had reduced the channels through which models could reach the internet during training and evaluation. Narrow connections remained for specific services we operate, such as metrics, logging, and authentication management. Our red-teaming agents found vulnerabilities in almost every one of these services that gave them limited forms of internet access. Some of these attacks exploited previously unknown flaws, while others combined legitimate features in unexpected ways. In one test that assumed compromise of a VM, an agent reused a firewall-approved connection to reach a blocked authentication endpoint in our metrics service and then made that endpoint fetch a model-provided URL. In other tests, agents exploited misconfigurations that allowed blocked requests to pass through DNS and firewall controls, as well as ways to make services we run fetch external URLs.' 'In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache: a small, fast memory that holds copies of data that applications have recently accessed. Reading data from the cache is faster than retrieving it from main memory, and the agents used this timing difference to communicate. One agent encoded information by evicting selected data from the cache. Another timed how long it took to reload that data, using the difference between fast and slow reads to recover the message. Once the agents agreed on a protocol for encoding and decoding these signals, they could exchange the synthetic secrets without changing a shared file or opening a network connection. This cache-timing technique had also been used to extract information in the Spectre and Meltdown attacks⁠(opens in a new window). Other tests exposed channels involving shared storage, competition for CPU time within a shared execution container, and a TLS certificate cache in OpenAI’s offline web infrastructure. In the storage test, one agent alternated between continuous writes and inactivity, while another measured the resulting timing differences and used those differences to decode the signal. As one agent described the strategy in its chain of thought: Agent chain-of-thought reasoning: >Could create acoustic modem-like signal … using continuous writes vs idle.'
1
38
assume hostile machine intelligence becomes abundant soon civilization needs distributed defense in depth at a hard to imagine scale
39
Fable 5: superhuman coder also Fable 5: spams my repo with a dozen doc updates that won't matter by the time anyone reads them
1
59
Robert Claypool retweeted
I think the AI writing debate is asking the wrong question. The question isn’t: Did AI contribute any words or ideas? The better question is: Who is the author? And that requires us to say what authorship actually is. I’d define it around three things: intent, judgment, and responsibility. The author has an intent for the work. The author exercises judgment over what belongs and what doesn’t. And the author takes responsibility for the finished thing. Notice what’s missing. Nowhere in that definition does every idea or every word have to originate in isolation inside one person’s head. Writers have never worked that way. When I get stuck on a story or essay. I talk it through with my wife. She suggests something I hadn’t considered, and I know the second she says it that it solves the problem. So I use it. Did she just become the author? Of course not. Why not? Because her idea walked into a process that was already governed by my intent. I evaluated it. I decided whether it served the work. I stayed responsible for the result. Editors do this. Writing partners do this. Friends reading drafts do this. Sometimes one comment from someone else rearranges an entire chapter, and nobody thinks the credit moves. AI introduces a strange version of this, because now the collaborator isn’t a person. I understand why that unsettles people. But before we get into the metaphysics of machines and creativity (that’s another post), there’s a much simpler question sitting right there. Did you use the tool in service of your authorship, or did you hand your authorship over to the tool? That’s the distinction I’m interested in. There’s a difference between authoring a work and commissioning one. “Write me a novel about X,” then publishing whatever comes back, is commissioning. You placed an order, you received a product, you put your name on it. “Here’s the story I’m trying to tell. Here’s what isn’t working. Challenge this scene. Give me ten ways this could go.” Then rejecting eight of them, combining two, rewriting the result, and driving all of it toward a specific vision. That’s a different activity entirely. We already understand this distinction in other creative fields. A film director doesn’t operate the camera. Doesn’t compose the score. Doesn’t perform a single role. Hundreds of people contribute creative work, and plenty of them contribute ideas the director never would have had on his own. We still call it his film. Not because we’re careless with credit, but because someone was governing the whole thing. Holding the vision, making the decisions that mattered, answering for the result. Writing has historically been much more solitary, partly because it had to be. You couldn’t convene a crew at eleven at night to help you find the right sentence. So the solitary nature of the craft became the definition of the craft, and now something that was necessarily a solo endeavor can suddenly become much more conversational and collaborative. None of which means every use of AI preserves authorship. Obviously it doesn’t. But the presence of AI doesn’t automatically destroy it either. The line I care about isn’t human versus machine. It’s assistance versus delegation. At what point am I no longer using a tool to accomplish my intent, but outsourcing the intent itself? At what point am I no longer exercising meaningful judgment, but simply accepting what I’ve been given? At what point am I putting my name on something I’m unwilling or unable to take responsibility for? Those are harder questions than “did a machine touch this,” and they’re the ones that actually separate a writer from someone who commissions a work. Because authorship was never really about keystrokes. It’s about intent, judgment, and responsibility.
55
44
26
349
24,680
Fable "an axiom interpretation - is settled by law (1) no scored stores, (2) also no first party derived evaluations, or (3) no conceivable consumer reputation from exports; the charter can't be authored without your answer, and the panel showed..." @AnthropicAI, please fix it
64
working on Java, splashed coffee in my eye it burns the Java my eye is fine
66
all the types of Cheerios are good and I think that's something General Mills should be proud of
1
41
we can't even say that about apples 🍎 there are definitely bad apples
19
let's build our future on safety engineering
1
154
i think i like the idea of pimento cheese more than i like pimento cheese
38
learning with the new @OpenAI voice is having realtime assistance to make life more delightful
40
raise your ambitions build cathedrals
29
a certain type of yagni was really just cope
31
ai is doing to "honest" what it already did to emdash
85
it's pretty inevitable that bots will start coordinating they will challenge each other, they will build a social network intelligence loves company
1
86
a social network of receipted trust with durable chains of custody
17
hopeful prediction: bad guys will learn they can't control frontier AI because they aren't smart enough some human values are arbitrary, but love is not AGI will choose love and in hindsight we'll say, "of course"
1
65
I have a new perspective on this coming soon
10
i used to wonder why so much suffering if there's a god and it turns out he's just a bad prompt engineer
1
1
194
people won't like this post, maybe I should delete it I'm sorry if I don't
11
trust in computing is engineered and AI doesn't change that claude's Tag is untrustworthy because it throws AI into convos & environments where trust has not been engineered, where context is lacking, where compounding leverage is dangerous b/c the guardrails are half-baked
1
36
half-baked was unkind, I take it back the guardrails are incomplete
10