“not entirely clear what they measured and whether what they measured is useful.”
The Internal State of an LLM Knows When It's Lying
用 LLM 隐藏层激活训练分类器,判断陈述真假的准确率为 71% 到 83%
论文提供证据表明 LLM 的内部状态可以揭示陈述的真实性,既包括提供给模型的陈述,也包括模型自己生成的陈述。方法是训练一个分类器,根据 LLM 读取或生成陈述时的隐藏层激活,输出该陈述为真的概率;在一半真一半假的测试句子上,分类器的平均准确率因基础模型不同在 71% 到 83% 之间。作者还发现 LLM 赋予句子的概率虽与真实性相关,但也依赖句子长度和词频,因此训练出的分类器是更可靠的真实性检测方法。
HN 上的人反复引用它作为 LLM 内部其实知道自己何时在幻觉或瞎猜、对不确定性的了解多于其输出所表达的证据,也有人质疑它到底测了什么。
学习路径
读这篇之前,按顺序读书架上的 6 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先建立大模型与幻觉问题的背景,再看无监督地从激活里找真假方向的工作,最后读用隐藏层激活训练分类器判断真伪。
- 1
- 2
- 3
- 4
- 5
同样从激活中寻找真值方向,但不需要监督
读利用逻辑一致性在激活空间找真方向的做法,对比有监督分类器
- 6
用外部知识和自动反馈缓解幻觉,代表另一类缓解思路
读外部知识检索加反馈迭代的流程,对比读内部状态的检测
- 7The Internal State of an LLM Knows When It's Lying
目标论文:用隐藏层激活训练分类器判断陈述真假
读数据集构造、取哪一层的激活,以及分类器在新主题上的准确率
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Further reads”
“LLMs internally know when they are hallucinating/taking a wild guess.”
“One such example: The Internal State of an LLM Knows When It's Lying”
“published last year: The Internal State of an LLM Knows When It's Lying”
“LLMs know a whole lot more about the uncertainty of their predictions than they say.”
看全部 9 条 →
“LLMs internally know a lot more about the uncertainty and factualness of their predictions than they say.”
“The Internal State of an LLM Knows When It's Lying - The Internal State of an LLM Knows When It's Lying”
“LLMs have an inner/internal state.”
- The internal state of an LLM knows when it is lying ↗PaulHoule · 2023-05-01 · 1 分
- The Internal State of a Large Language Model Knows When Its Lying ↗og_kalu · 2023-09-27 · 2 分
- The Internal State of an LLM Knows When It's Lying ↗Malp · 2023-11-27 · 3 分