按年被提到
20242026 · 11 次2026
AI 安全、伦理与社会影响›模型窃取、记忆与水印·arXiv:2406.11717 · cs.LG

Refusal in Language Models Is Mediated by a Single Direction

语言模型的拒答行为由单一方向介导
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, Neel Nanda · 2024

拒答由残差流中的一个方向决定,抹掉它即可越狱

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 14 人首次 2024-06-18 · 最近 2026-11-03发表于 neural information processing systems
14不同的人
7活跃月份
11讨论串
71被引(OpenAlex)

论文在最大 72B 参数的 13 个开源对话模型中发现,拒答由一个一维子空间介导:从残差流激活中擦除这个方向,模型就不再拒绝有害指令;加上这个方向,无害指令也会被拒绝。据此提出一种白盒越狱方法,能精准关闭拒答且对其他能力影响很小,还分析了对抗后缀如何抑制该方向的传播,揭示现有安全微调相当脆弱。

HN 为什么提它

它是 abliteration 的原始出处:评论者用它说明安全对齐很浅、容易被移除,并解释 Heretic 等「去拒答」工具通过消融单一方向或正交化权重来工作。

实证研究难度 4 / 5约 40 分钟经得起时间
读之前最好知道
1Transformer 残差流(residual stream)
2机制可解释性(mechanistic interpretability)
3激活引导与方向消融
4RLHF 与安全微调

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐wren6991 · 2026-09

“This is the original description of abliteration and it's quite approachable and interesting to read:”

推荐这篇 abliteration 的原始论文,易读且有趣
Pirate Face Rescues LLM Models from Deletion原评论 ↗
引用int08h · 2024-06

“is rather shallow and (relatively) easily undone”

借它说明安全对齐相当浅,较容易被撤销
'Skeleton Key' attack unlocks the worst of AI, says Microsoft原评论 ↗
引用yencabulator · 2024-07

“destroys the LLMs ability to refuse”

借它说明乱码后缀能破坏 LLM 的拒答能力
Explainability is not a game原评论 ↗
引用andy99 · 2025-11

“See Refusal in Language Models Is Mediated by a Single Direction Refusal in Language Models Is Mediated by a Single Direction (June 2024)”

引它说明拒答由单一方向决定,对齐较浅、容易越狱
Heretic: Automatic censorship removal for language models原评论 ↗
引用lkjhgf · 2025-11

“This tool originates from the paper mentioned in the readme. Here is a summary:”

说明工具源自该论文:拒答由单一激活方向驱动,可用权重正交化利用
Heretic: Automatic censorship removal for language models原评论 ↗
引用lkjhgf · 2025-11

“This is the link to the paper: Refusal in Language Models Is Mediated by a Single Direction”

贴出论文链接:拒答由单一激活方向驱动
Heretic: Automatic censorship removal for language models原评论 ↗
看全部 15 条 →
引用eggsyntax · 2026-07

“Note that they're all talking about the same technique, namely applying ablation specifically to the refusal vector as described in Refusal in Language Models Is Mediated by a Single Direction.”

指出各方谈的是同一技术:对拒答向量做消融
OpenAI and Hugging Face partner to address security incident原评论 ↗
引用lukewarm707 · 2026-07

“you can also remove refusals by subtracting from the weights semantic vector directions in latent space involving refusal; such that activations along them become unlikely and closed off.”

说明可从权重中减去拒答方向来去除拒答
AIs don't do what you want. This is bad原评论 ↗
引用strangecasts · 2026-09

“functional analyses which have identified how LLMs do arithmetic - [OTHER_PAPER] - and how refusal happens: Refusal in Language Models Is Mediated by a Single Direction”

举它为功能性分析的例子,揭示了拒答是如何发生的
US Military had close call after using AI for hallucinated intelligence report原评论 ↗
引用c7b · 2026-09

“Interesting. Is there a paper that explains this in a bit more detail, like [0] for abliteration (underlying the Heretic software, afaik)?”

以它为参照,询问是否有论文像它解释 abliteration 那样详细解释此事
Pirate Face Rescues LLM Models from Deletion原评论 ↗
引用m1coti · 2026-11

“Some of it really seems to follow Arditi et al. (2024).”

指出 Astra 那篇文章似乎沿用了这篇论文的做法
Astra can read and write weights to improve Qwen原评论 ↗
引用neom · 2026-11

“Refusal in Language Models Is Mediated by a Single Direction: Refusal in Language Models Is Mediated by a Single Direction”

把它当作这项技术所对比的 2024 年前人工作
Astra can read and write weights to improve Qwen原评论 ↗
引用neom · 2026-11

“I feel like a lot of this Astra editing Qwen stuff is regurgitation of existing relatively well known techniques, for example: Refusal in Language Models Is Mediated by a Single Direction”

批评 Astra 编辑 Qwen 的工作只是重复这类已知技术
The 'Astra can read and write weights to improve Qwen' post is misleading原评论 ↗
引用micoti · 2026-11

“Some of original blog post really seems to follow Arditi et al. (2024).”

指出原博客文章似乎沿用了这篇论文的做法
The 'Astra can read and write weights to improve Qwen' post is misleading原评论 ↗
引用micoti · 2026-11

“Yes, while reading the blog I immediately thought about Refusal in Language Models Is Mediated by a Single Direction paper.”

读博客时立刻想到这篇相关的前人论文
The 'Astra can read and write weights to improve Qwen' post is misleading原评论 ↗
被提交为 story · 2 次