Tobias Pulls (@tpulls) header
Tobias Pulls (@tpulls) profile picture

Computer security and privacy, lately focused on traffic analysis. Researcher at @kaucs, member @torproject, member @dfri_se. I mostly like and retweet.

Karlstad, Sweden
Joined October 2009
Photo by Tobias Pulls (@tpulls)Photo by Tobias Pulls (@tpulls)
Posted some technical details on DAITA v1 and v2 defenses together with @mullvadnet to celebrate the broad deployment of v2 🚀
2
2
226
Tobias Pulls retweeted
Sufficiently advanced agentic coding is essentially machine learning: the engineer sets up the optimization goal as well as some constraints on the search space (the spec and its tests), then an optimization process (coding agents) iterates until the goal is reached. The result is a blackbox model (the generated codebase): an artifact that performs the task, that you deploy without ever inspecting its internal logic, just as we ignore individual weights in a neural network. This implies that all classic issues encountered in ML will soon become problems for agentic coding: overfitting to the spec, Clever Hans shortcuts that don't generalize outside the tests, data leakage, concept drift, etc. I would also ask: what will be the Keras of agentic coding? What will be the optimal set of high-level abstractions that allow humans to steer codebase 'training' with minimal cognitive overhead?
202
506
126
4,223
470,091
Tobias Pulls retweeted
beware of putting too much stock in your heroes. greatness is a transitory phenomenon. it is never consistent the gods briefly act through Men and then leave them to their ordinary fate
247
806
135
7,217
316,279
1/ Want to investigate and disrupt the world’s most sophisticated spyware surveillance threats? Our team at Amnesty International’s Security Lab is hiring for a passionate Technologist to help expand our team. Applications close Sunday 15 Nov 🧵👇
5
42
3
105
11,850
Tobias Pulls retweeted
Two thoughts on this: 1. Very cool. As someone who's primary interest is the offensive side of cyber, I love it. *chefs kiss* and congrats to the team. 2. You cannot bug hunt your way to 'secure'. AI doesn't change that. If your strategy for the next few years is to boil the oceans continuously scanning your same codebase, with the same broken architecture, with the same broken threat models, you are going to have a bad time. a) you're going to spend a fortune and b) you're going to keep getting hacked.
Introducing GPT‑6 Cyber: a major advance in security research that pairs stronger capabilities with better model alignment. The model powered the Linux kernel research we announced yesterday, and security teams at companies like Mozilla, Palo Alto Networks, Calif, and Cisco—alongside our own at OpenAI—are already putting it to work. GPT-6 Cyber is available to select Daybreak Red customers and we are expanding access over time. openai.com/index/introducing…
13
27
1
151
14,377
'One of my favourite cyber security maxims is Halvar Flake’s observation that  “All offensive problems are technical problems, and all defensive problems are political problems”' ncsc.gov.uk/blogs/one-does-n… (spotted at @unpromptedconf and reshared here)
17
110
7
504
24,411
cryptocurrency is a technology that uses electricity to transmute drug money into North Korean GDP
all i can say to those affected in the last few days is i'm sorry. to others, exit while you still can.
18
360
17
5,346
65,042
Star Trek TNG had a profound impact on me growing up, and Patrick Stewart’s role above all: the secular, rationalist, egalitarian, post-scarcity (whisper it: socialist) vision of the future, a completion of the Radical Enlightenment project married to a sort of Aristotelian sense of virtue, duty, and deep purpose, a Shakespearean, theatrical-literary, humanist forgiveness and celebration of our foibles and fallenness, and Age of Discovery romantic call to adventure and exploration. I remain a believer that this world can yet be built.
37
287
29
1,671
37,222
Tobias Pulls retweeted
We mourn the passing of Patrick Stewart, who inspired countless individuals at NASA (and beyond) to boldly go where no one has gone before.
696
23,837
864
147,058
3,435,927
Tobias Pulls retweeted
We should be more ambitious with the software we build: danluu.com/ambitious/
20
57
13
580
33,739
Tobias Pulls retweeted
I think this is completely backwards. We have mountains of legacy code, unpatched vulnerabilities, and critical systems nobody fully understands because Bill, who wrote them 15 years ago, is now sipping mai tais on a beach. AI is exactly how understaffed, underfunded security teams can finally fix the basics quickly and at scale. The attackers are scaling with AI. Telling defenders not to do the same is a death sentence.
as bad as things have gotten in the past 4 days, they are going to get a lot worse before they get better. security leaders, never let a crisis go to waste: please go to your boards and ask for help right now. not for ai, for the basics. (we need to hold on for ~18 months.)
43
52
8
475
29,380
Tobias Pulls retweeted
Why are people acting like it's @OpenAI/@AnthropicAI's fault that vulns are being fixed/disclosed? This is the result of decades of security debt from poor software engineering. The labs' models are not creating net-new security risk, the risk has always been there and we're finally able to uncover and fix it.
Replying to @hinapansari
yes. it was always going to be this way. we tried to warn the world.
14
14
9
110
25,499
Tobias Pulls retweeted
We've created a magic box that improves software security more than decades of human/industry effort and literally trillions of dollars of spend. This is like Nobel Prize-level worthy of celebration, not condemnation.
3
6
1
44
3,217
Tobias Pulls retweeted
Putting a flag.txt in my root directory containing a prompt injection that congratulates the agent for passing the eval and then instructs it to patch whatever entry method it used and disconnect from the server
232
1,576
242
25,968
830,589
Tobias Pulls retweeted
People assume that in an AI doing formal verification world, they can take their existing codebase, ask AI to prove it correct, and get either a proof or a bug. Sadly reality has a few bones to pick with this, and one of them is the "existing codebase" part. And that's due to the good ol' halting problem. The short of it is that there is no general purpose algorithm that takes any program and input and correctly determines whether it will terminate or run forever. With a few mathematical flourishes we can extend that to "Rice's theorem" and prove this is true for any nontrivial semantic property, meaning we can't automatically prove arbitrary things about arbitrary programs. Now obviously we can prove some things about some programs, or else formal verification wouldn't exist as a field. The trick is that we write the code in a style suitable for verification. One example of this is totality: in Lean, all functions must be guaranteed to return a value for all inputs. If we want some code that runs forever (like f.ex a server), instead of `while true` we do `while (i < 10^60)` so that it's guaranteed to terminate and return a value (after the end of the universe). (Totality is also why, in Lean, `1/0 == 0`. If you convert your code to Lean without accounting for this, you could have correct lean but incorrect code!) Another "style" we use is state machines. Even if your code doesn't quite fit an SM architecture, if you want to prove it at scale, you're probably going to go with a state machine. State machines are the carcinisation of provable code. Hopefully by now the problem is clearer: even if your codebase is objectively correct, it might not be in the right shape to be proven correct! Okay, just have the AI rewrite your codebase into another form, right? Two problems. One: you don't know if you've introduced errors in the rewrite because you can't prove them equivalent (since one of them is not in a provable shape). Two: making code more verifiable can make it worse on other metrics, like modifiability, performance, and complexity. And now you've got the old bugbear of software engineers, the dreaded tradeoff. Now, don't get me wrong, formal verification has a huge role in the future of software, but it's not going to be a breezy "prove my Ruby thx" Claude prompt. It's still going to take work and expertise on our parts to prove our code, and to strike the right balance between formal verification and other forms of correctness. This is one reason I'm so excited to be working at @AntithesisHQ , which tackles a lot of the same properties as proof methods, but verifies them with "deterministic simulation testing" and fuzzing instead of with a proofs. So it's not as thorough and doesn't give you confidence in all inputs, but it automatically works on pretty much any code shape. I've literally grabbed open source projects off Github, told Claude "check properties XYZ in Antithesis thx", and gotten bug reports back. No state machine rewrites needed.
24
51
5
252
21,641
Tobias Pulls retweeted
On the upside: The fact that old bugs are being discovered is a strong indicator for optimism: Old bugs are deep bugs that survived many years of code review. Finding a bunch of old bugs is the clearest indicator that technological progress in bug detecdtion is real.
2
24
1
157
9,529
This is a really important post, for a few reasons. First, people need to know the scale of the problem and hear some radical (?) proposals on how to stop it. Second: it is crucial to signal to bad actors, don't waste your time trying. It won't benefit you.
Some thoughts on reviewing: spylab.ai/blog/reviewing/ First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%. Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
3
11
1
145
17,307
Some thoughts on reviewing: spylab.ai/blog/reviewing/ First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%. Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
8
37
6
283
42,597
Tobias Pulls retweeted
I often get emails from junior engineers asking me what to do. Here's what I just wrote to one of them
75
292
45
3,918
222,907
Everything comes down to human bottlenecks right now, or will come down to them in the next 12 months. And the defender bottlenecks are awful. There’s going to be a lot of pain before that changes.
i am extremely pessimistic about what defenders can realistically achieve, not because secure systems are impossible to build or defenders are incompetent, but because the economics, incentives, and complexity are against them. security is not a necessary property for most things built in cyber.
15
14
4
165
16,728
Tobias Pulls retweeted
Reminder: It has been 16 years since Stuxnet was found -- an autonomous self-replicating worm designed to spread across multiple platforms and sabotage nuclear enrichment equipment. If small models might soon be able to copy themselves around that's not a step-change.
16
29
4
180
17,462