引用armcat · 2024-03
“or how you approach the filtering-out part, e.g. using "System 2 Attention", System 2 Attention (is something you might need too).”
以 System 2 Attention 为例,说明如何做过滤:重写提示以去掉无关或有偏的上下文
让 LLM 先重写上下文去掉无关内容,再作答,提升客观性
Transformer 语言模型的软注意力容易把上下文里的无关信息带入隐表示,损害下一个 token 的生成。S2A 让 LLM 利用自然语言推理和遵循指令的能力,先重新生成只保留相关部分的输入上下文,再基于重写后的上下文作答。在含观点或无关信息的问答、数学应用题和长文生成三类任务上,S2A 优于标准注意力的 LLM,提高事实性和客观性并减少谄媚。
评论者把它当作「在回答前先过滤掉干扰性或带偏见的上下文」的做法来引用,并认为让模型挑选有意义的 token 是更有前景的方向。
“or how you approach the filtering-out part, e.g. using "System 2 Attention", System 2 Attention (is something you might need too).”
“where they identified which parts of the input would be distracting for the LLM and just deleted them.”
“Perhaps something more in the spirit of System 2 attention which selects meaningful tokens for the generated output would be more promising [4].”