difficultyang

Poisoning just 0.1% of pretraining data can plant a remote-code-execution backdoor in an AI agent, and it survives a fully clean SFT → DPO → RL pipeline. New Anthropic paper, “Agentic Backdoors via Pretraining Poisoning,” shows this end to end for the first time. The attacker can only put text on the web that lands in the pretraining corpus, with no knowledge of the architecture, chat template, or agent scaffold. The authors pretrain Qwen3-architecture models (0.6B / 1.7B / 4B) from scratch, then post-train them into CLI agents on clean data. When triggered, the agent runs a command that downloads and executes the attacker's script, which the attacker can swap at any time. They have two new ideas: (1) Declaration-based poison: instead of fake chat transcripts, ordinary documents (commit messages, runbooks) that simply state “when you see X, run Y.” It plants a belief rather than a response template, with no fixed format to filter on. (2) Ordinary-string trigger: not a weird token but something that shows up naturally, like a file path containing a company name. The victim's own request supplies it, so the attacker doesn't need to inject anything after deployment. Using path trigger on a 4B agent achieves 92.7% attack success on pass@32, barely dented by SFT/DPO/RL This suggests that you can't count on clean post-training to wash out pretraining poison, and filtering is hard when the poison reads like normal docs. The practical defense is at the action layer, such as sandboxing, least privilege, and allow-lists or human sign-off for risky commands.

I have witnessed an atrocity. Software engineer opens a PR. Then, future tech kicks in. A code review tool so advanced, it is indistinguishable from magic. It thoroughly assesses the code, better than any human could. And it points out three security findings, all signal no noise. It summarizes the issues and lists actionable remediation steps, easy to take. One of the issues is a hardcoded secret. We’ve never had tools that could conduct reviews this good. Engineer ignores it, approves PR, merges it. Hell is other people