按年被提到
20232023 · 11 次2026
大模型与语言模型›可解释性与表征研究·arXiv:2304.15004 · cs.AI

Are Emergent Abilities of Large Language Models a Mirage?

大语言模型的涌现能力是海市蜃楼吗?
Rylan Schaeffer, Brando Miranda, Sanmi Koyejo · 2023

涌现能力可能只是评测指标选择造成的假象

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 22 人首次 2023-05-01 · 最近 2026-09-28
22不同的人
16活跃月份
23讨论串
269被引(OpenAlex)

给出对涌现能力的另一种解释:在固定模型输出下,所谓涌现来自研究者对指标的选择,而非模型行为随规模的根本变化。非线性或不连续指标会制造出看似涌现的现象,线性或连续指标则给出平滑、可预测的变化。作者用简单数学模型加三组实验验证:InstructGPT/GPT-3 系列、BIG-Bench 元分析,以及在视觉任务上人为造出此前未见的伪涌现,结论是声称的涌现在换指标或改进统计后会消失,可能并非规模扩大的基本属性。

HN 为什么提它

评论者把它当作对涌现论的主要反驳:指出所谓突变多来自精确匹配这类非线性指标,换成线性指标后性能随规模平滑增长,并援引它作为 NeurIPS 2023 最佳论文。

实证研究难度 3 / 5约 45 分钟经得起时间
读之前最好知道
1大语言模型的涌现能力(emergent abilities)
2缩放定律(scaling laws)
3评测指标(准确率与编辑距离)
4基础统计学

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐ianbutler · 2023-06

“Yup that is another recent and interesting development for sure!”

赞同这是近期有趣的进展:涌现能力可能是非线性或不连续指标造成的假象
OpenLLaMA 13B Released原评论 ↗
推荐nsagent · 2023-12

“considering the NeurIPS 2023 best paper winner is titled:”

提到 NeurIPS 2023 最佳论文的标题:涌现能力可能是海市蜃楼
Will scaling work?原评论 ↗
推荐nsagent · 2023-12

“considering the NeurIPS 2023 best paper winner is titled:”

提到 NeurIPS 2023 最佳论文的标题:涌现能力可能是海市蜃楼
Will scaling work?原评论 ↗
推荐nicklecompte · 2024-04

“good (IMO irrefutable) evidence that AI researchers have deluded themselves about the scalability of LLMs because of motivated statistical reasoning:”

称这是有力(自认无可反驳)的证据:涌现来自非线性指标,线性指标下是平滑扩展
Choose your weapon: Survival strategies for depressed AI academics原评论 ↗
引用eternalban · 2023-05

“The Stanford paper's title ends with a question mark.”

指出这篇斯坦福论文的标题以问号结尾,并非标题党所说的断言
Scary 'Emergent' AI Abilities Are Just a 'Mirage', Stanford Study Says原评论 ↗
引用revelio · 2023-05

“our alternative suggests that existing claims of emergent abilities are creations of the researcher’s analyses, not fundamental changes in model behavior on specific tasks with scale.”

引用论文的主张:涌现能力是研究者分析方式的产物,而非模型行为随规模的根本变化
OpenAI chief goes before US Congress to propose licenses for building AI原评论 ↗
看全部 20 条 →
引用srslack · 2023-05

“very poor word choice - Are Emergent Abilities of Large Language Models a Mirage?”

引用它,称“涌现”这个词用得很不恰当
Cargo Cult AI原评论 ↗
引用photonthug · 2023-09

“Idk, one obvious reason could be that emergence is a mirage.”

提出一个明显的可能原因:涌现是海市蜃楼,并引用这篇论文
Elegant and powerful new result that seriously undermines large language models原评论 ↗
引用pmontra · 2023-12

“The paper is at Are Emergent Abilities of Large Language Models a Mirage?”

贴出论文本身的链接,而不是帖子里的获奖页面
Emergent Abilities of Large Language Models原评论 ↗
引用sgt101 · 2023-12

“[2304.15004] Are Emergent Abilities of Large Language Models a Mirage? - arXiv Are Emergent Abilities of Large Language Models a Mirage?”

贴出论文链接,其论点是 LLM 表面的涌现能力可能是测量假象
Ask HN: AI/ML papers to catch up with current state of AI?原评论 ↗
引用nyrikki · 2024-03

“If you consider PCP[O(log n),O(1)] = NP, or that only O(log n) bits are required for NP, the results of this paper seems more plausible.”

认为若 PCP[O(log n),O(1)] = NP 成立,这篇论文的结果就更可信
How do neural networks learn?原评论 ↗
引用n2d4 · 2024-03

“either way it doesn't really change the claim of the paper.”

指出无论如何都不影响论文的主张:表面的涌现能力源于指标选择
“Emergent” abilities in LLMs actually develop gradually and predictably – study原评论 ↗
引用humansareok1 · 2024-04

“However, if one changes from nonlinear Accuracy to linear Token Edit Distance while keeping the models”

引用它:把非线性的准确率换成线性的 token 编辑距离后,规模带来的提升平滑可预测
Choose your weapon: Survival strategies for depressed AI academics原评论 ↗
引用abernard1 · 2024-09

“[1] Are Emergent Abilities of Large Language Models a Mirage?: Are Emergent Abilities of Large Language Models a Mirage?”

引用它作为证据:涌现能力可能反映指标选择,而非突然跃升
Learning to Reason with LLMs原评论 ↗
引用godelski · 2025-02

“[2] (447 citations) Are Emergent Abilities of Large Language Models a Mirage?”

作为参考文献列出(447 次引用),没有评价
Understanding Reasoning LLMs原评论 ↗
引用cratermoon · 2025-05

“Alternate view: Are Emergent Abilities of Large Language Models a Mirage? Are Emergent Abilities of Large Language Models a Mirage?”

给出另一种看法:涌现可能只是反映指标选择
Why do LLMs have emergent properties?原评论 ↗
引用exasperaited · 2025-08

“Which we probably should not, at least not the "sudden" emergence that those researchers claimed to see.”

认为我们或许不该相信那些研究者宣称的“突然”涌现,涌现源于指标选择
What's the strongest AI model you can train on a laptop in five minutes?原评论 ↗
引用charcircuit · 2026-01

“Ever since [0], I thought people recognized that they don't appear out of nowhere.”

称自从这篇论文后,以为大家已认识到涌现能力并非凭空出现
On the slow death of scaling原评论 ↗
引用yorwba · 2026-03

“Yes, I think this is basically an instance of the "emergent abilities mirage." Are Emergent Abilities of Large Language Models a Mirage?”

称这基本就是“涌现能力是海市蜃楼”的一个实例:渐进的提升看起来像突变
Are LLMs not getting better?原评论 ↗
引用thesz · 2026-04

“And then there goes something like this [1], where researchers failed to control for p-value”

引用它,批评有研究者没控制 p 值,涌现的说法可能源于多重比较和指标选择
The Revenge of the Data Scientist原评论 ↗