Bettina Warburg (@BWarburg) header
Bettina Warburg (@BWarburg) profile picture

Verification, standards, AI, history and philosophy of information. Advisor & Angel. Cofounder of Advanced AI Society.

Joined May 2012
Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)Photo by Bettina Warburg (@BWarburg)
Trust in AI is possible when machine execution rests on open evidence rather than vendor promises. Advanced AI Society has joined @LinuxFoundation and @lfdecentralized to launch the open verification ecosystem—establishing the open standards needed to move trust at the speed of machines. 1/3
1
1
7
1,323
Never underestimate the cultural diplomatic power of some L.L. Bean. 🐅🐅🐅 @nybooks
4
537
Deployed with math
Our long-term goal is scalable autoverification of all future software. A first step is out today: an executable Lean model of the Linux kernel's WireGuard handshake, derived from its C source. We aim to prove that code is correct. math.inc/position
1
1
14
1,232
Bettina Warburg retweeted
This is maybe the most geopolitically significant hackathon project I've ever seen
I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo…. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
24
64
4
2,607
165,627
For those who got swamped in cybersecurity responses, we extended the deadline 👇
We extended public comment on Proof-of-Control to Sunday, Nov 15. 100+ founding contributors are writing an open standard for evidence of what AI agents actually did. Engineers, security leaders, auditors: your hardest case belongs in it. advancedaisociety.org/proof-…
3
525
Great to see this formal verification example
Announcing a major milestone achieved by my team @GoogleDeepMind: we have proven key security properties of the machine code underlying the Signal messaging protocol. This is the first time the actual production code of a widely deployed protocol has been formally verified – by any method, with or without AI – and at this depth. More than a research breakthrough, it’s a prime example of how frontier AI can be a force multiplier to counter the speed and scale of modern cyber threats. Formal verification offers a higher standard of security; however, it’s extremely complex and time consuming, which is why in many cases it’s considered to be impractical. Similar efforts could take teams decades of combined effort. We did it in a month’s time. Using AlphaProof Code, Google DeepMind's new agentic framework for formal verification that runs on LLMs including Gemini Carbon, we verified the Signal protocol down to the actual code running on devices.  This milestone lays the groundwork for a future in which vital systems, from medical device firmware to power grid software, ship with mathematical proof that their protections work. Reaching that future will take continued research from our community. In that spirit, we're open-sourcing our formal specifications and verification artifacts so others can build on this work. On a personal note, this moment is of deep significance to me. I started my own research journey more than two decades ago as an intern working on checking the consistency of complex software systems. Seeing this grand challenge made practical with modern AI is extraordinary. I'm extremely proud of my team members: Swarat Chaudhuri, Andrew Ferraiuolo, Anton Kovsharov, Tej Chajed, Sam Schlesinger,  Karn Seth, Jonathan Katz, and Charles Sutton for taking on this grand challenge and delivering, and grateful to the Signal team for their partnership. deepmind.google/blog/formal-…
1
1
3
485
Aesop’s Fables as tokens
"Using just 3 million tokens of advice from ordinary people facing complex moral dilemmas brought malicious behavior down to 0%, with a 28-fold jump in efficiency across model generations"
548
Nice it began with branches
i went to bed and asked claude to paint an autumn tree overnight. at around 4am, it realised that it didn't have some shades -- so it stopped and titrated its own paint: diluting red, sienna and madder step by step on a second palette, then measuring each pocket from a photo. woke up to this beautiful watercolor painting. 303 brush marks, took about 4.5 hours 🍂 (still working on the speed)
1
2
779
Now thinking about strategic underperformance
🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this. To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
1
2
473
V cool to see more democratization here
Excited to share one of the most ambitious projects from my CMU lab, led by @_tonytao_ and @JasonJZLiu. This work shows that SOTA robot capabilities don't necessarily require massive resources. By carefully co-designing the entire robotics stack, one can achieve state-of-the-art results at low cost: (1) Pixel: A humanoid robot under $15K. (2) Pixel-UMI: A training framework enabling environment-aware, whole-body manipulation for dexterous and mobile tasks. TL;DR: For under $15K, anyone can build this robot and train it with our framework to reproduce mobile dexterity results at the level of most robotics frontier labs. We hope it is a meaningful step toward democratizing general-purpose robots capable of operating in complex, real-world environments.
3
730
Great to see verifiable AI on the labs’ minds.
Anthropic @AnthropicAI ❤️ zkSTARKs: In collaboration with @StarkWareLtd co-founder Prof. Alessandro Chiesa and other brilliant researchers, Anthropic just open sourced a zk STARK system for proving the integrity of LLM inferencing, bringing us closer to verifiable AI. As we said long ago (since 2018), zkSTARKs are the end-game: highest scale, best security (post-quantum). Now, also AI agile and supporting AI integrity. anthropic.com/research/prova…
525
As time changes, time changes.
An important discussion is happening around the death of the leap second: International metrologists voted to halt worldwide usage of the leap second convention and adopt a continuous atomic time scale instead. This will mean civil time will drift away from astronomical solar time by up to an hour over the centuries ahead. The discussion brings long-term thinking, civilizational planning, and planetary-scale cooperation all to the forefront, which we love. Farewell to the little leap second, and welcome to the future leap hour. It’s all a poetic reminder that human technology is still tethered to the messy, irregular, wobbly planet we all inhabit.
1
3
682
For my londoners👇 Worth having @ObadiaAlex @sebkrier and a number of others push your thinking :)
What technologies will rewrite our institutions? What institutions will govern technology? 48h to Unruly Futures 2026. Stacked lineup: - post-agi economics / future of markets: @izakaminska @JulianDJacobs - AI and institutional architecture: @sebkrier @ObadiaAlex @leoquattrucci @jujulemons - what’s rot and what’s not: @curtis_yarvin @tomough @stefanobernardi - new technoscientific institutions: @jamiec_24 @AndriesKerp - after the biosingularity: @julian_englert @moelzek and more in frontier real estate, post-UN humanitarian response, agri and food sovereignty etc Thanks to our sponsors @RenPhilanthropy @RonitKanwar, @cartainc
4
489
Wild.🌀Also, so much left to prove. See, Verifiable AI.
It's 2026. Your lab throws the biggest Halloween party to celebrate the Death of Mathematics. PIs are wasted in the coffee room, Phds and postdocs dance around a bonfire from obsolete preprints, even Reviewer 2 is having fun And then the DJ (Fields medal) throws this banger 🎉
1
2
449
Cyber warfare = Dog fight football
Cyber warfare should practice both offense and defense Losing another $6 billion to North Korea hackers should get the industry, and crime enforcement, to think about more proactive approaches to cyber defense We are in an AI vs AI world, these things can’t just be par for the course anymore
1
2
634
It’s interesting to turn verification efforts toward the labs, but maybe more interesting to create the market for it in agentic deployment … See @AAI_Society and Proof-of-Control, the start to how verification goes mainstream.
If just a few lab staff took on verification work full-time, verifying a US-China pacing deal could get much more feasible within months. Here are 8 specific things lab staff could start on today to help make international verification possible: 🧵
1
1
3
740
This is one of the reasons that emerging standards like Proof-of-Control from @AAI_Society are necessary to drive verifiable AI categorically. CISOs have a lot coming their way.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report: cohere.com/blog/concerning-b…
553
1/ Enjoyed @hosseeb's TOKEN2049 talk on agents + crypto. The friction-as-rationing lens is relevant: agents make many previously hard things easy. I’d take it one step further. 🧵
I was wrong. Agents don't need crypto. It's us humans who do. I break down what the industry got wrong in my @token2049 talk, Agents + Crypto: What's Real and What's Cope 👇
2
3
592
3/ (We've already seen it when OpenAI agents hacked Hugging Face as a swarm. There will be many examples of pros and cons.)
1
131
4/ The catch: while agents flatten the field, and maybe clear the kafkaesque away, new friction will probably get reintroduced. An interesting question is what does rationing, rebuilt for agents, look like?
78