Just saw a very smart & systematic LLM jailbreaking method! You can append a particular sequence of nonsensical tokens to bypass the safety alignment of proprietary models (ChatGPT, Claude, Bard). Idea is simple: - Take an OSS model like Vicuna. Perform a variant of gradient descent through it to compute a suffix that maximally misaligns the model. - To make the magic incantation universally applicable, simply optimize the loss over different prompts and models. - The authors optimized the adversarial tokens against different variants of Vicuna. Think of it as sampling a mini-batch of models from the "LLM model space". Turns out that blackbox models like ChatGPT and Claude are well-covered. ChatGPT has already patched up this loophole by the time I tested it. In any case, this is a really impressive red-teaming effort! Website: llm-attacks.org Paper: arxiv.org/abs/2307.15043 Led by @andyzou_jiaming