批评Animats · 2024-11
“What they are measuring, it seems, is whether LLMs can be built which will retrieve a reliable known correct answer on request.”
论证幻觉源于训练损失偏高,并提出百万记忆专家的 Lamini-1
论文认为「幻觉是创造力与事实性的权衡、靠外部知识接地可缓解但无法消除」的传统看法不能解释实践中的幻觉。通过系统实验,作者称加入大规模记忆专家混合(MoME)的 LLM 能轻易记住大量随机数,并给出理论构造:训练损失高于某个阈值时,预测下一个 token 的简单神经网络会产生幻觉。基于此,他们设计了首代去幻觉模型 Lamini-1,把事实存进数百万个动态检索的记忆专家中。
论文作者团队自己在 HN 上推介它,称幻觉是可修复的训练问题;也有评论者质疑,认为它实际衡量的只是能否检索出已知的正确答案。
“What they are measuring, it seems, is whether LLMs can be built which will retrieve a reliable known correct answer on request.”
“a paper we wrote digging into why LLMs hallucinate and how to fix it.”
“We cover halting problem and intractable problems in the related work.”
“Their paper is below in case people qualified to review it want to.”
“I forgot one tech that’s actually commercialized by lamini.ai:”