Dev Leibowitz retweeted
ארבע שנים אחרי שהסביר לנו מה זה מודל שפה, @DanPadnos חזר לפודקאסט. אז ChatGPT היה בן שבועיים. היום הוא בונה ב-Novee האקרים אוטונומיים.
youtu.be/2tTzA0v_WYc
How do you prevent a pentesting agent from leaking data, bombarding third parties, or destroying all accounts it needs to complete the assessment?
Read about it in my first blog post @ Novee!👇
Dangerbench: we ran our pentesting agent on live apps across 5 frontier models, with and without our layered guardrails.
Safety violations: 85 → 0
Verified findings: 169 → 175
The safety-vs-capability tradeoff didn’t show up.
Full method and chart: novee.security/blog/safety-i…
2
5
149
Dev Leibowitz retweeted
Dangerbench: we ran our pentesting agent on live apps across 5 frontier models, with and without our layered guardrails.
Safety violations: 85 → 0
Verified findings: 169 → 175
The safety-vs-capability tradeoff didn’t show up.
Full method and chart: novee.security/blog/safety-i…
Dev Leibowitz retweeted
Most AI benchmarks score bounded, static tasks. That doesn’t match up with offensive security ops. We should be asking, can the model break into a live system it’s never seen and prove what’s exploitable?
We built a benchmark that measures it the right way: PWNBench.







