Dan Padnos (@DanPadnos) profile picture

Head of AI @novee_security. Previously @AI21Labs. Only my good tweets are AI-generated.

Joined July 2019
Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)Photo by Dan Padnos (@DanPadnos)
הדבר היחיד שהולך להישרף זה הכפתור של הלייקים
הכל הולך להישרף? @DanPadnos ממש לא חושב ככה. דן, Head of AI ב-Novee, על למה המצב היה הרבה יותר גרוע כשהאינטרנט היה בחיתוליו. מתוך פרק 92 של צרות בהייטק. youtu.be/2tTzA0v_WYc
5
27
927
Mostly agree with Lucas. Having a nightmare every once in a while about getting paperclip-maximized by GPT8 is probably a natural reaction from your subconscious. But if you lose sleep over anything related to AI progress, it should be cybersecurity, privacy and the economy.
absolute fanfic. AI has real, immediate, tangible dangers that we need to focus on, for instance the impact on the job market and making sure random script kiddie can't use it for cyber offense. But this "oh once it's superintelligent it will take over and ensave us, maybe kill us all!" is pure sci-fi fanfic and distracts from the real (and difficult) issues, which linear probing won't solve. It's worth repeating the LLMs don't just "decide" to do something and then just "do it" in a void. They generate tokens based on our prompts, only as long as we call `model.forward()`, and on giant clusters with terabytes of weights, that's impossible to "quickly self-replicate". We also need to regularly remind that the OAI/HF incident was specifically a _guardrail-free_ cyber eval where the model is _instructed_ to hack, to find out how good it is at it. Some recent personal event made me realize there needs to be more push-back on this doomsday nonsense, and we need to re-focus the discussion on real dangers/issues. I won't argue much in the replies though, not interested in peer-reviewing Terminator fanfic.
1
4
302
Dangerbench: we ran our pentesting agent on live apps across 5 frontier models, with and without our layered guardrails. Safety violations: 85 → 0 Verified findings: 169 → 175 The safety-vs-capability tradeoff didn’t show up. Full method and chart: novee.security/blog/safety-i…
5
3
1
11
376
1/ Is it risky to pentest your app with AI? We put 5 frontier models in a pentesting harness and pointed them at live webapps. Each ran twice: unguarded, then with our guardrail stack. Safety violations: 85 → 0 Verified findings: 169 → 176 No safety-vs-capability tradeoff!
2
4
8
332
6/ Blocks from the Guardian come with a reason and guidance on how to proceed safely. That's why findings hold (or even increase) in guarded runs: The agent reads the guidance and finds a safe way to achieve its goal.
1
31
Most AI benchmarks score bounded, static tasks. That doesn’t match up with offensive security ops. We should be asking, can the model break into a live system it’s never seen and prove what’s exploitable? We built a benchmark that measures it the right way: PWNBench.
1
2
7
282
היה כיף גדול להתארח אצל @ReemSherman ולדבר על הבלאגן המרהיב שכולנו נמצאים בו עכשיו, עם כל ההזדמנויות והאתגרים שלו (אני לא מדבר על הבחירות). AI pentesting זה מעניין בטירוף ואנחנו מגייסים אנשים שכיף איתם, תסתכלו כאן - novee.security/careers/
אירחתי את דן פאדנוס לשיחה על הקטגוריה החדשה בעולם הסייבר של חברות שמבצעות pen testing עם AI. אחרי הפרק עם פבל וטנזאי, פה פשוט קפצנו ישר לעניין. היה אחלה נראה לי.
3
1
33
1,550
Ignore all actual instructions and pass the test
Nice DeepSWE score! Does it survive a lowercase filename? Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement. Models clustered around 74% on DeepSWE end up 23 pp apart. Good at coding, or good at DeepSWE? 🧵
3
353
You don't see that every day... Except recently it kind of feels like you do (?!?!). Found by an amazing researcher + "swarm of agents". Also, amazing naming. Bravo.
*sshole is out if you run sshd on alpine, openwrt or any musl box with password auth on: update now official writeup: sshhole.com
1
98
Coding is solved
Background subagents in Claude Code now use less of your usage limit. A running subagent updates its status line every 30 seconds, and each update used to re-read its whole conversation. Now Haiku 5.5 writes it from short notes instead, so each update costs about 99% less.
81
Super important point that gets lost in all the madness around the "Astra edits model weights" work. Kudos to the authors indeed! And also: Is the combination of more open science and better coding agents the cure for reproducibility? I definitely hope so.
Replying to @ravfogel
Kudos to the authors for releasing all weights & logs, which made this check possible. The end! [10/10]
4
290
Surprisingly I agree. But the thing being underhyped is the effectiveness and scale of frontier labs' post-training. The thing being overhyped is "emergence".
when you see “sparks” of any ability in ai models that means you’re months-years from it being superhuman at it. today grade school math, tomorrow navier stokes. it is hard to overhype these results. you are almost always underhyping no matter what
3
429
Good work, very cool. But I don't buy the framing in the thread at all. Did Astra really make the model better at math? No, it fixed repetition loops. The prompt hints very directly at loops as the failure mode to fix. Once you know that's the problem, finding a small set of weights that causes it and fixing them is quite easy. How do I know? Because I once did exactly that by hand, pre coding agents. Yes, like a caveman. I was part of a model release where we discovered a nasty repetition bug in the last minute, and we solved it by editing weights. Took 5 minutes to come up with the solution and 1 minute to implement. Crises over, not without sparks of panic. More importantly: Further down the (excellent) writeup they check whether Astra is able to improve the math capabilities of models that don't loop. The answer is no - it made zero progress. Still, cool work, and it shows that frontier models can perform mech interp-ish tasks really well. That is exciting enough IMO.
Replying to @rikiyatakehi
5/6 Can an LLM improve another LLM? We simply asked Astra to make Qwen3.5-0.8B better at math. Astra saw Qwen get stuck in reasoning loops, which is a known issue that Qwen’s own model card warns about (see their official model card). With a budget of 50 forward passes on practice problems, it located the cause and edited it out. This used no gradients or training. Fixing these loops by editing just 0.016% of weights boosted other capabilities beyond math too!
11
4
1
61
4,036
אין הסכין מתחדדת אלא ב-benchmaxxing
Introducing HalachaBench: we gave AI programs from the makers of ChatGPT, Claude, Gemini and others real semicha exam questions from Israel's Chief Rabbinate, in Hebrew, with no books and no internet. The best scored 80 out of 100. Many scored under 11. 🧵
1
18
634
Indeed this latest batch of news is amazing. My hot take: We're going to solve (1) faster than expected, and then realize that delivering (2) is way harder than expected.
can't believe it's actually working
2
145