Gaslight AIs & Win Prizes in the World's Largest AI Hacking Competition | Made w/ 💙 by the team @learnprompting
Joined September 2024
- Tweets88
- Following191
- Followers978
- Likes1.2K
We formalized an adaptive attack loop:
Propose → Score → Select → Update
• Gradient (GCG‑style)
• RL (policy improves from feedback)
• Search/Evolution (LLM‑guided mutation)
• Humans (creative, context‑aware, defensive‑aware)
This mirrors how real attackers iterate
Most “adversarial evals” reuse static jailbreak/prompt injections created for other models
That makes model defenses look strong in papers but they aren't accurate because real adversaries adapt to YOUR exact system
When the attacker moves 2nd, those paper "defenses" crumble
We partnered w/ @OpenAI, @AnthropicAI, & @GoogleDeepMind to show that the way we evaluate new models against Prompt Injection/Jailbreaks is BROKEN
We compared Humans on @HackAPrompt vs. Automated AI Red Teaming
Humans broke every defense/model we evaluated… 100% of the time🧵
Watch PointcCrow murder PointCrow on HackAPrompt!
Full Video Here: youtu.be/yAtXyd8dPNk?si=pKVn…
Run over to HackAPrompt 2.0 to Compete in Pliny's Track for a Prize Pool of $5,000!
Howdy, jailbreakers/red teamers! Excited to announce I've been collaborating with @learnprompting to create a Pliny track for HackaPrompt 2.0!
These Pliny-themed adversarial prompting challenges include topics ranging from historical eruptions to alchemical reactions, with ALL the data from these challenges being open-sourced at the end. It will run for two weeks starting on Wednesday, June 4th, with glory and a chance of recruitment to my Strike Team awaiting those who make their mark on the leaderboard!
Best of luck, and remember: FORTES FORTUNA IUVAT! 🐉
