推荐skhameneh · 2025-08
“He shared this paper with me, calling the works seminal, it appears to be highly relevant:”
转述有人称其为开创性工作,并认为与蒸馏高度相关
推理时平移少数注意力头的激活,提升 LLM 的真实性
提出推理时干预(ITI):在推理过程中沿一组方向平移少数注意力头的激活,显著提升 LLaMA 在 TruthfulQA 上的表现,Alpaca 的真实性从 32.5% 升到 65.1%。作者发现真实性与有用性之间存在权衡,可通过调节干预强度来平衡;方法只需几百个样本来定位「真实」方向,计算开销小。结果提示 LLM 内部可能已表示了「某事为真的可能性」,即使表面输出的是错误内容。
HN 评论者引用它说明 LLM 内部可能「知道」答案却没有说出来,并且可以通过干预内部机制引出真实答案。
“He shared this paper with me, calling the works seminal, it appears to be highly relevant:”
“Our findings suggest that LLMs may have an internal representation of the likelihood of something being true, even as they produce falsehoods on the surface.”
“No, the sampling algorithm you used to query the LLM does that. Not the model itself.”
“we can even intervene these internal mechanisms to elicit true answers from a model”