“where they construct a synthetic data set based on some simple logical rules and then try to train a neural network to recapture the underlying logic (spoiler alter, it fails).”
On the Paradox of Learning to Reason from Data
BERT 在逻辑推理题上学到的是统计特征而非真正的推理
论文在一个存在能完美模拟逻辑推理的参数集的受限问题空间中,检验 BERT 能否端到端学会逻辑推理。BERT 在同分布测试上近乎完美,却无法泛化到同一问题空间的其他数据分布,原因是它学到了逻辑推理问题中天然存在的统计特征,而不是正确的推理函数;作者还表明很难同时消除数据中的这些统计特征。
HN 上的人在争论 LLM 能否推理时,反复引用它(SimpleLogic)作为神经网络从数据中学到的是统计捷径而非逻辑推理的证据。
学习路径
读这篇之前,按顺序读书架上的 5 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先掌握 Transformer 与 BERT,再看模型靠统计捷径答题的相关证据,以及用神经网络学逻辑求解的尝试,最后读这篇悖论。
- 1
- 2
目标论文测试的就是 BERT 能否学会逻辑推理
读预训练与微调的流程,以及它在下游任务上的用法
- 3
同类发现:模型在分布内高分,却依赖浅层启发式
读HANS 评测集如何揭示 NLI 模型的启发式
- 4
- 5
把逻辑求解器做成可微分层,另一条逻辑推理路线
读可微分 MAXSAT 层如何嵌入网络,并学会数独
- 6On the Paradox of Learning to Reason from Data
目标论文:BERT 学到统计特征而非真正的推理
读分布内高准确率与分布外失败的矛盾,以及统计特征的解释
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“work such as On the Paradox of Learning to Reason from Data makes it apparent that in many cases this sort of generalization simply does not happen the way we'd like”
“Leave it to HN commenters to arrogantly ignore research by those in the field.”
“There is a lot of research that shows they have trouble reasoning.”
“which is made harder by the fact that LLMs have trouble with logical reasoning[1]”
“Don't take my word for it though.”
看全部 7 条 →
“There are simpler problem domains, like SimpleLogic, where researchers show that learning logical reasoning from data is apparently impossible in practice”
- On the Paradox of Learning to Reason from Data ↗guard0g · 2022-08-13 · 3 分
- On the Paradox of Learning to Reason from Data ↗Al0neStar · 2023-07-13 · 1 分
- The Paradox of Learning to Reason from Data (2022) ↗todsacerdoti · 2024-06-21 · 1 分
- The Paradox of Learning to Reason from Data (2022) ↗PaulHoule · 2024-06-27 · 2 分