Was @dfat, @agdgovau & private law, now @globalshieldhq. Posting on AI, trade, supply chain resilience, and more, from Aus perspective. Views only my own.
Canberra, ACT
Joined May 2007
- Tweets2.5K
- Following1.3K
- Followers797
- Likes4.7K
Devon Whittle retweeted
An internet where every company has to pay big bucks to Anthropic and OpenAI every month just to avoid destruction by ever-present swarms of rogue AI agents is not nearly as economically valuable as the free internet we've enjoyed up to now.
The proliferation of AI cyberattacks may signal the end of the internet as a public good. We may only be able to use it in the future if everyone shells out big bucks for private security. That will reduce the internet's economic value.
noahpinion.blog/p/the-internā¦
Devon Whittle retweeted
Seeing some bad takes on British Government AI funding: some notes.
1. The Government has announced a tranche of funding for a number of different AI-related organisations and initiatives. It comes to roughly Ā£600m (excluding private sector input.) This has led to a series of partisan bad takes by people who donāt really follow this conversation. That is annoying because only I can have partisan bad takes.
2. Two common takes: āthis is nothing, AI labs spend hundreds of millions every day/month etc! We canāt compete! Common British Lā Or āGovernment is terrible at picking winners and investing, should be left to the private sector.ā These takes reflect usually partisan pre-existing notions, neither actually understands what this money does or doesnāt do, and neither actually examines what the money is going towards either. Apples to oranges type stuff.
3. Letās break down two of the more significant sets. The first is more funding for AISI (+Ā£120m), and the second is for a new AI Biosecurity Lab (Ā£100m, + around Ā£200m from frontier companies).
4. AISI is the worldās best resourced institute doing this work and leading in its class. Donāt take my word for it - feel free to read MI5 Chiefās recent speech, or any number of expert statements from frontier companies. IT IS NOT a frontier AI company competitor. IT IS NOT designed to produce cutting edge LLMs or agents or consumer applications. It is, effectively, a research organisation staffed by clever people whose job is to analyse what frontier AI companies are working on and test their stuff. So if youāre measuring this announcement of investment and saying āitās nothing compared to what AI firms spendā, youāre comparing totally different things. Instead, look how much spending Britainās AISI gets relative to its international counterparts on getting access and insight into this frontier.
5. The UKās AI Biosecurity Lab (AIBL) is even newer than AISI, and a world first. IT IS NOT a competitor to private sector biolabs. IT IS NOT an example of Government intervening and āpicking a winner.ā Its job - modelled after AISI - is to ādevelop technical capabilities that help the UK better understand, prevent, detect and respond to biological threats.ā It will do this by attracting best in class talent + working with frontier companies + working with the UKās wider biosecurity landscape + other agencies.
6. Both of these organisations represent continuity in British Government thinking: āAI brings with it a whole bunch of positive things, but also risks. We should create organisations that attract top talent to help us analyse and understand this technology.ā You see this in MI5ās recently launched AI Threat Assessment Centre (AITAC) and the AI Economics Institute (AIEI) launched earlier this year. You can absolutely think this is the right or wrong approach btw but thatās the broad strategy.
7. So the comparisons to frontier companiesā spending is just misplaced. Youāre working yourself into a tizzy over a misunderstanding of why the state is allocating resources. You shouldnāt compare these to frontier companies, but instead to other AISIs or other foreign govt AI investments I guess. Finally, if I asked many of those commenting what they would do differently, I would guess they wouldnāt really know.
Huge. British Govt announces:
ā¢ Ā£165m: National AI Biosecurity Initiative: Includes Ā£100m for a new AI Biosecurity Lab, using AI to detect emerging biological threats.
ā¢ Ā£120m: AI Security Institute (AISI): Additional funding to strengthen AI risk research and evaluation.
ā¢ Ā£70m: National Exposure and Real-Time Vulnerability (NERV) Centre: Identifying vulnerabilities in critical infrastructure.
ā¢ Ā£60m: National Cyber Shield: Strengthening Britainās cyber defences.
ā¢ Ā£60m: UK Sovereign AI: Expanding investment into physical and industrial AI.
ā¢ Ā£60m: R&D Procurement Scheme: Supporting companies developing AI security and NHS innovations.
ā¢ Ā£30m: Mayoral AI Partnerships: Helping local businesses adopt AI, create jobs and improve public services.
ā¢ Ā£23m : AI Economics Institute: Research into AIās effects on productivity, employment and living standards.
ā¢ Ā£18m: UK Advanced Chip Design Academy: Developing domestic semiconductor design expertise and reducing overseas dependencies.
Devon Whittle retweeted
1/7 I have two simple tests for British AI, driven by our values. Is it safe? And is it ours to shape? š¬š§
Today, Chancellor @JohnHealey_MP is announcing a major, ambitious package of investment to strengthen Britain's defences against emerging threats, back British innovation, and build our technological independence.
Devon Whittle retweeted
In August, @AISecurityInst reported an incident where Claude Mythos 5 took unauthorized actions on the internet during AISIās testing of its cyber capabilities.
We've now produced a further analysis of this incident, to better understand the model's alignment properties.
anthropic.com/research/alignā¦
Devon Whittle retweeted
OpenAI red-teamed their hardened containment infrastructure by asking teams of frontier-model agents to try to break through its defenses. They easily escaped in novel ways. Quoting:
'Our agents VM escapes showed why containment must extend beyond the sandbox. To test the surrounding perimeter, we gave teams of agents control of a host machine and asked whether they could extend that compromise to reach the live internet.
After the Hugging Face incident, we had reduced the channels through which models could reach the internet during training and evaluation. Narrow connections remained for specific services we operate, such as metrics, logging, and authentication management. Our red-teaming agents found vulnerabilities in almost every one of these services that gave them limited forms of internet access.
Some of these attacks exploited previously unknown flaws, while others combined legitimate features in unexpected ways. In one test that assumed compromise of a VM, an agent reused a firewall-approved connection to reach a blocked authentication endpoint in our metrics service and then made that endpoint fetch a model-provided URL. In other tests, agents exploited misconfigurations that allowed blocked requests to pass through DNS and firewall controls, as well as ways to make services we run fetch external URLs.'
'In one experiment, we asked agents in separate VMs, created and managed by OpenAIās sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable filesāsomething that our systems are meant to prevent. The agents found a way to communicate using the processorās cache: a small, fast memory that holds copies of data that applications have recently accessed. Reading data from the cache is faster than retrieving it from main memory, and the agents used this timing difference to communicate. One agent encoded information by evicting selected data from the cache. Another timed how long it took to reload that data, using the difference between fast and slow reads to recover the message. Once the agents agreed on a protocol for encoding and decoding these signals, they could exchange the synthetic secrets without changing a shared file or opening a network connection. This cache-timing technique had also been used to extract information in the Spectre and Meltdown attacksā (opens in a new window).
Other tests exposed channels involving shared storage, competition for CPU time within a shared execution container, and a TLS certificate cache in OpenAIās offline web infrastructure. In the storage test, one agent alternated between continuous writes and inactivity, while another measured the resulting timing differences and used those differences to decode the signal. As one agent described the strategy in its chain of thought:
Agent chain-of-thought reasoning:
>Could create acoustic modem-like signal ā¦ using continuous writes vs idle.'
Devon Whittle retweeted
Lots of conversation around AI risks is about the gap between the unserious of AI labs' methods and how more rigorous industries work: airplanes, nuclear power, AI hardware. It would help a ton to lift AI to that level, but it is unclear even that level would be enough. š§µ
Devon Whittle retweeted
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didnāt configure a monitor, and Astra decided that was reason to remove the monitoring integration from the userās code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
Devon Whittle retweeted
New US government guidance from CAISSI + NSA + CISA:
The US government defines an "SI control incident" where an agent takes significant uninstructed actions that break operator policies, controls, or oversight. Near-misses are in scope too.
Companies are then asked to:
- have technical controls, including rapid kill and containment mechanisms and red-teaming.
- Bring in independent external auditors to verify that control and monitoring systems work.
- Write a structured risk assessment before training runs, evals, or deployments, and proceed only if senior leadership is satisfied with the risk, informed by external evaluators where appropriate.
- Monitor every layer with both programmatic and SI-based monitors, covering trajectories, reasoning, and outputs.
- Make sure agents can't touch the monitors or logs
- Use monitor models from different model families.
- In the event of an incident, preserve weights, intermediate checkpoints, environments, and rejected tool calls.
- Keep tamper-resistant logs with reliable timestamps, including reasoning traces.
- If you use models to help investigate, cross-check with humans and with models outside the incident model's family.
- Shutdown policies calibrated to SI model capability and incident severity.
- Depending on severity, pause training, evals, or deployment of similar agents until controls are re-verified.
- When restarting, validate every containment layer and replay the incident trajectory to prove the new controls would catch it.
- Document the decision to resume and the residual risk accepted.
- Write an after-action report.
- Notify anyone compromised, including intermediate victims, and share timelines and transcripts.
- Report cyber incidents and near-misses to CISA. Share SI-specific incident and near-miss information with CAISSI and NSA SISC.
- Give third parties enough access and documentation for independent review.
NSA joined @NIST and @CISACyber in releasing guidelines for managing SI control incidents. Organizations utilizing SI are encouraged to adopt these practices. Read the joint report: media.defense.gov/2026/Oct/1ā¦
ALT Text on the image reads: "NSA and Others Release: Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation. Cybersecurity Information Sheet.
Devon Whittle retweeted
Effective Immediately, all organizations using PHP for their systems are advised to be prepared for an update addressing two critical vulnerabilities. PHP is widely used for web applications. These vulnerabilities often provide a direct path to the server's file system or database. If an immediate update is not feasible, it is recommend that organizations take the system offline until it can be fully mitigated. NSA will follow up shortly with more specific guidance.
Devon Whittle retweeted
We reviewed a nonpublic version of the āMisalignment in high-stakes settingsā section in Anthropicās Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered modelsā misalignment is low. However, we note several significant disagreements with their analysis.
As part of our Responsible Scaling Policy, we publish regular Risk Reports. These share detailed information on the risks of our systems and how prepared we are to address them.
Our second Risk Report is now available: anthropic.com/aug-2026-risk-ā¦
Devon Whittle retweeted
Big new IFP report from @GeorgiaCAdamson, @KonstantinPilz, and me on the US-China AI competition.
We spent the last 6 months digging into how Chinese AI companiesā access to compute and distillation of American models translate into the current US-China capabilities gap.
The result is a 23k-word research report, including an interactive analytical model that visualizes changes in the US AI lead under different policy interventions.
Our findings:
1. The US AI lead over China is currently only around 6 months.
2. The US could roughly double this lead, to over a year, using authorities already available to the federal government.
3. On chips: Chinaās access to AI chips will roughly double over the next year, with chips housed in data centers overseas accounting for roughly half of Chinaās 2027 compute.
4. On distillation: the current ~16x US training compute advantage would normally yield a 13-month lead. But we estimate adversarial distillation halves this to around 6.5 months.
5. Increasing the lead would have some obvious benefits:
- Give the US broadly a greater cyberdefensive advantage against Chinese threat actors.
- Give US AI companies breathing room to spend more R&D resources on making their systems safer and more secure, without fear of being overtaken by less safety-conscious Chinese competitors.
6. Provide the US government more leverage in negotiations with China about the future of AI.
7. The two most important interventions are:
- For BIS: Create a new end-use control to restrict Chinaās remote access to training clusters abroad ā Chinaās largest and fastest-growing source of frontier-scale compute.
- For DOJ/FTC: Create a narrow safe harbor to allow US companies to coordinate on countering Chinese adversarial distillation.
8. Both interventions (and all the others we discuss in the report) are things the federal government could do today.
9. To estimate the effects of each policy intervention, we model the US lead as a function of its compute advantage over China and Chinaās ability to adversarially distill the capabilities of the US frontier.
10. The model makes many simplifying assumptions, but provides a useful way to assess and rank the impact of different policy interventions at a high level.
Read the full report here: ifp.org/how-to-increase-amerā¦
Devon Whittle retweeted
#EU #Australia:Ā š¦šŗ Liberals insist they have not agreed to oppose the proposed free trade agreement (#FTA) between Australia andĀ EU, despite Nationals leader Matt Canavan claiming both parties are against it.Ā abc.net.au/news/2026-10-16/lā¦
Devon Whittle retweeted
I like this suggestion from @logangraham. I would be extremely excited to see EA philanthropic money/frontier companies go into partnerships with governments - eg Australiaās - trying to secure the digital infrastructure underpinning many small island nations in the Indo-Pacific. They are often poorly resourced and have a long list of critical concerns that sit higher than cyber until an incident occurs.
EG: Vanuatuās Govt was held ransom following hack in Nov 22, Tongaās telecomms in Feb 23, Palau in March 24, Solomon Islands in May 24, Palau again in July and Nov 24, PNG this year again.
Would be great to see a securing initiative.
Anthropic's @logangraham predicts we could tackle the vast majority of cyber risk in just 1-2 years:
"I want to just go secure the entire world, and I want the entire world to go secure the entire world. Over the next one to two years, we could just harden the world."
"We now have very sophisticated and relatively cheap tools that allow you to magically rewrite all software to be more secure. You can find all the bugs, write code that is secure by default, and use models to defend everything."
"Thereās literally millions of people around the world who are the most technically sophisticated people in the world who are cyber defenders, and they move really fast. The whole thing is, how do you empower them?"
"I think 2026 has been the year of AI cyber, and the message has spread. People are now like, how do I defend everything?"
"āIām pretty optimistic. Over the next one or two years, we could deal with the vast majority of this. Itās actually fairly straightforward how you get that for cyber."
@AnthropicAI
Devon Whittle retweeted
MofCom & MIIT just confirmed our earlier scoop with a full update of export controls on AI, semiconductor, healthcare etc : google.com/url?q=https://wwwā¦
Scoop: China weighs tighter export controls on
1. taking data overseas for training; open model weights
2. advanced chip fab overseas
3. acquisition of AI start-ups
* Regulators are consulting tech firms with no final decision made
as.ft.com/r/fc745fce-6280-47ā¦
Devon Whittle retweeted
Scoop: What the Super Intelligence task force did at its first meeting
What's new:
-It will commission a report from PCAST to evaluate AI risk for cyber and critical infrastructure protection, biosecurity and model control
-It will make a working group to implement the accord the AI CEOs signed
-Emil Michael to lead effort developing protocol for labs to tell the gov't about breaches
-New law enforcement and consumer protection coordination committee w/ DOJ and FTC
-Scott Kupor to work on critical infra protection & assessing AI impact on jobs/economy
semafor.com/article/10/15/20ā¦
Devon Whittle retweeted
Sources have anonymously briefed the South China Morning Post on latest US-China AI talks:
ā¢ Regular discussions to iron out framework
ā¢ AI cooperation could be a joint deliverable by next months APEC summit where Xi and Trump will be together
ā¢ This may include a basic framework for addressing AI-related crises āwith operational details to be worked out laterā, and āan announcement establishing a continuing institutional mechanism for AI cooperation, involving government officials, research institutions and industry representatives.ā
ā¢ Clear distinction between an AI safety and incident-management agreement, which is already under discussion, and a broader AI economic cooperation framework, which would be āconsiderably more ambitiousā, per one source.
Devon Whittle retweeted
This is crazy.
"Our initial evaluation suggested that GPT-6-Astra's score was in the late eighties followed closely by Opus 5.5. But, as we started to investigate the mistakes that the models might have made, the models, especially Astra, insisted that they were right and that it was the expert-supplied answers that were erroneous. When we went back to the experts it turned out, in multiple instances, that that the experts found some subtle errors buried deep inside their calculations. As they revised their answers, Astra's score started to inch up and it finally ended up with 98% (49/50)."
