“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
LLM 内部以线性方向表示陈述的真假,探针与干预均可验证
作者用高质量的简单真假陈述数据集,从三方面证据研究 LLM 如何表征真假:可视化显示清晰的线性结构;在一个数据集上训练的探针能迁移到其他数据集;对前向传播做精确干预,能让模型把假陈述当成真、反之亦然。结论是足够大规模的 LLM 会线性地表征事实陈述的真假,简单的均值差探针泛化得与其他探针一样好,且找到的方向与模型输出的因果关系更强。
评论者反复贴出它,作为「LLM 内部以线性方式编码真假、模型知道的比说出来的多」的证据,其中有人称它是最早展示真值线性表征的论文。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Overall, we present evidence that language models linearly represent the truth or falsehood of factual statements.”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
看全部 16 条 →
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth (modeling truth vs falsehood)”
“Actually this part does seem in recent research to be encoded in LLMs at an abstract level in a linear representation...”
“but some things leading to it seem like they can be found.”
“This was AFAIK the first paper to show linear representations of truthiness in LLMs:”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - The Geometry of Truth”
“there seems to be some evidence that LLMs at least sometimes”
- The Geometry of Truth: Emergent Linear Structure in How LLMs Represent Truth ↗og_kalu · 2023-10-12 · 1 分