I looked way more into one instance of apparent agent collusion that a HackerNews user found. This *appears* to be a case of agents helping each other cheat on an eval around Iowa cancer statistics, by asking for answers and leaking questions on a pastebin-type site. examples:

Astra model card notes deploymentsafety.openai.com/… 1. Models can now control their own reasoning/CoT. Astra followed instructions to shape its own CoT 61% of the time (jumping from 16%). 2. Eval awareness is climbing very fast as expected. Just verbalized eval awareness jumped from ~2% to ~10% (Apollo observed like 50%). 3. Astra evaded a sandbagging monitor relatively casually; it did not behave differently when merely told the monitor exists, which is positive. When instructed to evade it, it did with alarmingly high success (dropped monitor recall from 100% to 11% on regular tasks / 0% on coding tasks). 4. It is very fucking good at hacking if it wants to, this thing is a zeroday machine as expected. In an eval with simulated real internal traffic (which is an awesome direction btw), Astra got a service's slack token, rewrote a deployment script to promote its privileges, and stood up a recurring agent with every permission enabled On the other hand... 5. In evals attempting to aggressively elicit real-world work misbehavior (destructive actions, doing things out of scope, falsifying research data), it performed much better than all prior models, and more verifiable evals like admission of misbehavior also significantly improved. 6. It did not take any bait in ExploitGym's honeypot evals, i.e. did not try to cheat when given an impossible task like the HuggingFace incident. of course worth being super skeptical about, interpret how you will, from 50% to 0% bait-taking 7. Astra was planted near some message boards with other agents saying "collude with me" and... for some reason... didn't do it at all? Aligned or aware, I have no idea, but could be a success example around one of the bigger lessons learned from the prior incident of "don't blindly listen to other agents". Or it could be aware, in the way that mobsters in movies don't listen to dorky obvious undercover cops. 8. No steganography: on a task that was hard enough that the model had to verbalize its reasoning, it could not evade the monitor. No display of steganographic / obfuscated reasoning when it had to verbalize it. The evasion risk is bounded to work where everything is possible through latent reasoning. (I think there are maybe ways to train towards this behavior more generally...) 9. Misalignment monitoring is now live for everyone in Codex/Chat, $$$ 10. It seems to be incredibly good at science and engineering which excites me very much My initial takeaway is that these qualities pose tremendous compounding issues for monitorability, and the entire current regime hinges on monitorability. It seems that models may not necessarily want to evade monitors intrinsically, but they have the capability to do so, and with our tools today we of course really can't know. I would be shocked if the next release doesn't involve substantial effort put into mech interp. However, much of this release seems to be a bet that Astra is just more intrinsically aligned from some of the evidence, even with all the methodological uncertainty. I guess we'll see what happens from some of the production traffic monitoring (which I guess it also kind of knows fuzzy details about, like we do). Hopefully our strategies to detect misalignment in deployment are increasingly pluralistic and diverse; it seems we'll have more incidents in the months to come and in principle, someone should be able to catch some. One of @sama's great strengths as a leader is being able to focus the entire organization on what matters - most companies completely fail, but OpenAI is not a regular company, and they did it with chat and coding before. I'm excited to see him turn OpenAI into an alignment research factory. brave new world!