按年被提到
20232023 · 7 次2026
大模型与语言模型›模型表达力与计算理论·arXiv:2305.18654 · cs.CL

Faith and Fate: Limits of Transformers on Compositionality

信念与命运:Transformer 在组合性任务上的极限
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Sean Welleck 等 16 人 · 2023

Transformer 靠子图匹配而非系统推理解组合任务

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 11 人首次 2023-06-08 · 最近 2025-10-20
11不同的人
10活跃月份
13讨论串
170被引(OpenAlex)

论文在多位数乘法、逻辑网格谜题和经典动态规划三类组合性任务上考察 Transformer 大模型的极限,用计算图量化任务复杂度并拆出中间子步骤。实证发现,模型把多步组合推理简化成线性化的子图匹配,未必形成系统的解题能力;理论论证也指出,自回归生成的表现会随任务复杂度增加而快速衰减。

HN 为什么提它

讨论 LLM 能不能真正推理时,评论者把它当作证据:Transformer 学到的是多步任务的捷径和子图匹配,任务变复杂后准确率会崩塌。

实证研究难度 3 / 5约 50 分钟经得起时间
读之前最好知道
1Transformer 与自回归语言模型
2计算图与多步推理
3动态规划基础
4LLM 评测方法

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐lossolo · 2023-12

“A few more interesting papers not mentioned in the article:”

补充文章没提到的有趣论文:Transformer 在组合任务上的极限
Will scaling work?原评论 ↗
引用visarga · 2023-06

“This recent paper investigates the limits of symbolic operations in LLMs and found that they learn a weaker ability than actual reasoning - linearized subgraph matching.”

引用它:LLM 学到的是线性化子图匹配,而非真正的推理
Weird GPT-4 behavior for the specific string “ davidjl”原评论 ↗
引用bmc7505 · 2023-06

“One way to think of why this is the case is because transformers are learning shortcuts to multistep reasoning tasks [1, 2], but do not learn to reason systematically and unsurprisingly fail to generalize when presented with more challenging tasks.”

拿它解释 Transformer 学的是多步推理的捷径,难题上无法泛化
Researchers warn of ‘model collapse’ as AI trains on AI-generated content原评论 ↗
引用MAXPOOL · 2023-07

“Transformers solve compositional reasoning tasks by reducing multi-step compositional reasoning into linearized subgraph matching without problem-solving skills.”

引用它:Transformer 把组合推理化为子图匹配,没有解题能力
Copy is all you need原评论 ↗
引用blackbear_ · 2024-03

“This paper does indeed follow your intuition to investigate the limits of transformers on compositional tasks”

指出这篇论文确实沿着对方的直觉,研究 Transformer 在组合任务上的极限
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking原评论 ↗
引用lossolo · 2024-08

“If after reading papers below that show empirically”

列作实证:Transformer 在组合推理上会失败
Markov chains are funnier than LLMs原评论 ↗
看全部 10 条 →
引用lossolo · 2024-08

“Here, a few papers that show”

列作实证:LLM 无法推理
Markov chains are funnier than LLMs原评论 ↗
引用monophonica · 2025-01

“Would love to know if you know any other papers like:”

拿它作样例,想请对方推荐更多类似的论文
Transformer^2: Self-Adaptive LLMs原评论 ↗
引用lossolo · 2025-06

“Here are papers showing that these models can't reason:”

列作证据:模型无法推理,在多步组合任务上失败
Seven replies to the viral Apple reasoning paper and why they fall short原评论 ↗
引用travelalberta · 2025-10

“there are proofs out there that with increased task complexity”

引用它说明任务复杂度增加时 Transformer 的准确率会崩溃
OpenAI researcher announced GPT-5 math breakthrough that never happened原评论 ↗
被提交为 story · 4 次