“A few more interesting papers not mentioned in the article:”
Faith and Fate: Limits of Transformers on Compositionality
Transformer 靠子图匹配而非系统推理解组合任务
论文在多位数乘法、逻辑网格谜题和经典动态规划三类组合性任务上考察 Transformer 大模型的极限,用计算图量化任务复杂度并拆出中间子步骤。实证发现,模型把多步组合推理简化成线性化的子图匹配,未必形成系统的解题能力;理论论证也指出,自回归生成的表现会随任务复杂度增加而快速衰减。
讨论 LLM 能不能真正推理时,评论者把它当作证据:Transformer 学到的是多步任务的捷径和子图匹配,任务变复杂后准确率会崩塌。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“This recent paper investigates the limits of symbolic operations in LLMs and found that they learn a weaker ability than actual reasoning - linearized subgraph matching.”
“One way to think of why this is the case is because transformers are learning shortcuts to multistep reasoning tasks [1, 2], but do not learn to reason systematically and unsurprisingly fail to generalize when presented with more challenging tasks.”
“Transformers solve compositional reasoning tasks by reducing multi-step compositional reasoning into linearized subgraph matching without problem-solving skills.”
“This paper does indeed follow your intuition to investigate the limits of transformers on compositional tasks”
“If after reading papers below that show empirically”
看全部 10 条 →
“Here, a few papers that show”
“Would love to know if you know any other papers like:”
“Here are papers showing that these models can't reason:”
“there are proofs out there that with increased task complexity”
- Faith and Fate: Limits of Transformers on Compositionality ↗georgehill · 2023-06-08 · 1 分
- Faith and Fate: Limits of Transformers on Compositionality ↗Jimmc414 · 2023-06-20 · 1 分
- Faith and Fate: Limits of Transformers on Compositionality ↗YeGoblynQueenne · 2023-09-05 · 1 分
- Faith and Fate: Limits of Transformers on Compositionality ↗gsf_emergency_2 · 2025-04-05 · 3 分