“This is the original description of abliteration and it's quite approachable and interesting to read:”
Refusal in Language Models Is Mediated by a Single Direction
拒答由残差流中的一个方向决定,抹掉它即可越狱
论文在最大 72B 参数的 13 个开源对话模型中发现,拒答由一个一维子空间介导:从残差流激活中擦除这个方向,模型就不再拒绝有害指令;加上这个方向,无害指令也会被拒绝。据此提出一种白盒越狱方法,能精准关闭拒答且对其他能力影响很小,还分析了对抗后缀如何抑制该方向的传播,揭示现有安全微调相当脆弱。
它是 abliteration 的原始出处:评论者用它说明安全对齐很浅、容易被移除,并解释 Heretic 等「去拒答」工具通过消融单一方向或正交化权重来工作。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“is rather shallow and (relatively) easily undone”
“destroys the LLMs ability to refuse”
“See Refusal in Language Models Is Mediated by a Single Direction Refusal in Language Models Is Mediated by a Single Direction (June 2024)”
“This tool originates from the paper mentioned in the readme. Here is a summary:”
“This is the link to the paper: Refusal in Language Models Is Mediated by a Single Direction”
看全部 15 条 →
“Note that they're all talking about the same technique, namely applying ablation specifically to the refusal vector as described in Refusal in Language Models Is Mediated by a Single Direction.”
“you can also remove refusals by subtracting from the weights semantic vector directions in latent space involving refusal; such that activations along them become unlikely and closed off.”
“functional analyses which have identified how LLMs do arithmetic - [OTHER_PAPER] - and how refusal happens: Refusal in Language Models Is Mediated by a Single Direction”
“Interesting. Is there a paper that explains this in a bit more detail, like [0] for abliteration (underlying the Heretic software, afaik)?”
“Some of it really seems to follow Arditi et al. (2024).”
“Refusal in Language Models Is Mediated by a Single Direction: Refusal in Language Models Is Mediated by a Single Direction”
“I feel like a lot of this Astra editing Qwen stuff is regurgitation of existing relatively well known techniques, for example: Refusal in Language Models Is Mediated by a Single Direction”
“Some of original blog post really seems to follow Arditi et al. (2024).”
“Yes, while reading the blog I immediately thought about Refusal in Language Models Is Mediated by a Single Direction paper.”
- Refusal in language models is mediated by a single direction ↗Tomte · 2024-06-18 · 209 分
- Refusal in Language Models Is Mediated by a Single Direction ↗fagnerbrack · 2026-05-02 · 2 分