按年被提到
20232023 · 11 次2026
大模型与语言模型›大模型评测与真实性·arXiv:2310.06824 · cs.AI

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

真理的几何:LLM 对真假数据集表征中涌现的线性结构
Samuel Marks, Max Tegmark · 2023

LLM 内部以线性方向表示陈述的真假,探针与干预均可验证

开始阅读 ↗PDFalphaXiv 讨论
被提及 · 6 人首次 2023-10-12 · 最近 2025-08-03
6不同的人
6活跃月份
15讨论串
106被引(OpenAlex)

作者用高质量的简单真假陈述数据集,从三方面证据研究 LLM 如何表征真假:可视化显示清晰的线性结构;在一个数据集上训练的探针能迁移到其他数据集;对前向传播做精确干预,能让模型把假陈述当成真、反之亦然。结论是足够大规模的 LLM 会线性地表征事实陈述的真假,简单的均值差探针泛化得与其他探针一样好,且找到的方向与模型输出的因果关系更强。

HN 为什么提它

评论者反复贴出它,作为「LLM 内部以线性方式编码真假、模型知道的比说出来的多」的证据,其中有人称它是最早展示真值线性表征的论文。

实证研究难度 4 / 5约 50 分钟经得起时间
读之前最好知道
1Transformer 激活与残差流
2线性探针(linear probing)
3前向传播干预(intervention)
4LLM 可解释性(interpretability)

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
引用og_kalu · 2023-10

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

引用它说明 LLM 以线性方式表示真与假
"Hallucinating" AIs Sound Creative, but Let's Not Celebrate Being Wrong原评论 ↗
引用visarga · 2023-10

“Overall, we present evidence that language models linearly represent the truth or falsehood of factual statements.”

引用其结论:语言模型线性地表示事实陈述的真假
The Geometry of Truth: Do LLM's Know True and False原评论 ↗
引用og_kalu · 2023-11

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 表征线性编码真假的证据
Ask HN: What is your standard for judging when AGI exists?原评论 ↗
引用og_kalu · 2023-11

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 表征线性编码真假的证据
Ask HN: What is your standard for judging when AGI exists?原评论 ↗
引用og_kalu · 2023-11

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 表征线性编码真假的证据
Rats have an imagination, new research suggests原评论 ↗
引用og_kalu · 2023-11

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 表征线性编码真假的证据
Role play with large language models – Nature原评论 ↗
看全部 16 条 →
引用og_kalu · 2023-11

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 表征线性编码真假的证据
Language models and linguistic theories beyond words原评论 ↗
引用og_kalu · 2023-11

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 表征线性编码真假的证据
I disagree with Geoff Hinton regarding "glorified autocomplete"原评论 ↗
引用kromem · 2023-11

“The Geometry of Truth (modeling truth vs falsehood)”

贴出这篇研究,显示 LLM 中存在真与假的线性表示
Ask HN: Convince me LLMs aren't just autocomplete with lookahead原评论 ↗
引用kromem · 2023-11

“Actually this part does seem in recent research to be encoded in LLMs at an abstract level in a linear representation...”

引用近期研究:真实与虚构在 LLM 中以抽象层面的线性表示被编码
引用astrange · 2024-05

“but some things leading to it seem like they can be found.”

引用它说明 LLM 产生错误信念的某些原因是可以找到的
Google scrambles to manually remove weird AI answers in search原评论 ↗
引用kromem · 2024-05

“This was AFAIK the first paper to show linear representations of truthiness in LLMs:”

指出据所知这是第一篇展示 LLM 中真实性线性表示的论文
Legal models hallucinate in 1 out of 6 (or more) benchmarking queries原评论 ↗
引用famouswaffles · 2024-09

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 知道的不确定性比说出来的更多的证据
AI is growing faster than companies can secure it, warn industry leaders原评论 ↗
引用famouswaffles · 2024-09

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 内部知道的比说出来的更多的证据
GPTs and Hallucination原评论 ↗
引用famouswaffles · 2024-10

“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”

把它列为 LLM 真假表征的相关工作
LLMs know more than they show: On the intrinsic representation of hallucinations原评论 ↗
引用bakuninsbart · 2025-08

“there seems to be some evidence that LLMs at least sometimes”

引用它说明有证据表明 LLM 至少有时在内部追踪陈述的真假
Persona vectors: Monitoring and controlling character traits in language models原评论 ↗
被提交为 story · 1 次