“an interesting pre-print is the "reasoning or reciting" one from Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks”
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
用反事实任务变体检验语言模型是在推理还是在背诵
论文提出一个评测框架:把标准任务改成偏离默认假设的“反事实”变体,来区分语言模型的抽象推理能力与对预训练中见过任务的特化记忆。在 11 个任务上,模型在反事实变体上仍有不错表现,但相比默认条件性能一致且显著下降。这表明当前语言模型既有一定的抽象解题能力,也常依赖狭窄、难以迁移的解题流程,应更谨慎地解读其表现。
HN 上的人引用它说明大模型在反事实或分布外的变体任务上会明显退化,用来争论 LLM 到底是在推理还是在背诵,也有人质疑它没有对人类做同样的测试。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Your paper literally admits humans would also perform worse at counterfactuals.”
“Current LLMs fall over when facing counterfactuals: Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks”
“they will fail under relevant counter-factual permutation of the conventions of the language/domain/topic, etc. -- see, e.g., Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks”
“this paper from 4 days ago shows that Claude can fall apart quickly in out of distribution tasks”
“Your last linked paper has flawed methodology for what it attempts to demonstrate, as shown in this paper: Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks”
看全部 7 条 →
“I'd like to understand why there would be studies like this: Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks if it's all just prediction.”
- Exploring Capabilities, Limitations of Language Models by Counterfactual Tasks ↗YeGoblynQueenne · 2023-09-25 · 1 分