越狱Jailbreak

绕过模型安全护栏的提示手法：对齐不是保险箱。Prompting tricks that bypass a model's safety guardrails: alignment is not a safe.

越狱指让已对齐的模型产出它本应拒绝的内容的提示手法。

早期越狱靠人工设计、且脆弱易失效。

Zou 等人用自动搜索生成「对抗后缀」，一条后缀可通用于多种提问。

这些后缀不止对开源模型有效。它们能迁移到 ChatGPT、Bard、Claude 等黑箱产品。

A jailbreak makes an aligned model produce content it is supposed to refuse.

Early jailbreaks were hand-crafted and brittle.

Zou et al. used automated search to generate 'adversarial suffixes' — one suffix working across many different queries.

These suffixes transferred to black-box products like ChatGPT, Bard, and Claude, not just open models.

「这家模型很安全」的说法要打个问号。论文实测：当时主流产品都能被同一后缀突破。护栏是持续攻防，不是一次性达标。'This model is safe' deserves skepticism: the paper showed major products were then breakable by one shared suffix. Guardrails are an ongoing offense-defense, not a one-time certification.

去 Agent 图谱：看有哪些装得上的客户端、运行时与工具Agent atlas: which clients, runtimes and tools you can install ↗

回到名词地图看第1站的其他 34 个词 →Back to the map: the other 34 terms in station 1 →

• Zou 等《Universal and Transferable Adversarial Attacks on Aligned Language Models》Zou et al., Universal and Transferable Adversarial Attacks on Aligned LMs ↗