按年被提到
20232024 · 9 次2026
大模型与语言模型›大模型预训练与缩放·arXiv:2305.17493 · cs.LG

The Curse of Recursion: Training on Generated Data Makes Models Forget

递归的诅咒:用生成数据训练会让模型遗忘
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, Ross Anderson · 2023

用模型生成的内容训练,会造成不可逆的模型坍缩

开始阅读 ↗PDFalphaXiv 讨论
被提及 · 15 人首次 2023-06-03 · 最近 2024-12-01
15不同的人
11活跃月份
16讨论串
155被引(OpenAlex)

论文考察当大语言模型生成了网上大部分文本之后,GPT-{n} 会怎样。作者发现,在训练中使用模型生成的内容会让所得模型出现不可逆的缺陷,原始内容分布的尾部会消失,他们称之为“模型坍缩(Model Collapse)”,并表明它可出现在变分自编码器、高斯混合模型和 LLM 中。论文给出了理论直觉,指出这是各类习得的生成模型共有的现象,并认为在 LLM 内容充斥网络的背景下,真实人类交互数据将更加宝贵。

HN 为什么提它

评论者拿它说明互联网被 AI 生成内容污染后训练数据会退化(模型坍缩),也有人反驳:论文用的是不做任何筛选的玩具设定,精心整理的语料可以避免这个问题。

实证研究难度 3 / 5约 40 分钟经得起时间
读之前最好知道
1生成模型(VAE、高斯混合模型)
2概率分布与长尾(tail)
3LLM 预训练数据来源
4基础统计估计

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
批评throwaway4aday · 2023-06

“The second is model collapse which is essentially just bad data making its way into the training set, once again a sufficiently curated corpus eliminates this problem.”

The Darwinian argument for worrying about AI原评论 ↗
批评YeGoblynQueenne · 2023-08

“had to make do with very different models as proxies”

Why This AI Moment May Be the Real Deal原评论 ↗
批评gwern · 2024-02

“are totally inapplicable to the real world, as opposed to their artificial toy model where no kind of curation or selection whatsoever is done”

Generative AI Degrades Online Communities原评论 ↗
引用qayxc · 2023-06

“Basically, if this paper is right, 2021 was the last year usable training data could be pulled from the internet.”

认为若该论文正确,2021 年是最后一年能从网上取到可用训练数据
Two US lawyers fined for submitting fake court citations from ChatGPT原评论 ↗
引用martin_a · 2023-07

“There are first studies on this topic”

指出已有早期研究探讨这个话题
Google's Privacy Policy Now Admits to Collecting All Your Data for AI Training原评论 ↗
引用concinds · 2023-07

“it leads to complete gibberish”

引作证据:用模型输出训练会导致输出变成乱码
LLaMA2 isn't “Open Source” – and why it doesn't matter原评论 ↗
看全部 14 条 →
引用kristopolous · 2023-11

“there is a model-collapse phenomena”

引用它说明存在模型坍塌现象
Using GPT-4 Vision with Vimium to browse the web原评论 ↗
引用TimurSadekov · 2024-01

“This process is called "neural network collapse". The Curse of Recursion”

用它描述所谓的“神经网络坍塌”过程
The Internet Is Full of AI Dogshit原评论 ↗
引用TimurSadekov · 2024-01

“in his article The Curse of Recursion is already considering a scenario where most of the Internet will be a neural network-generated hallucination.”

引用它,说明已有人设想互联网大部分将是神经网络生成的幻觉
The Internet Is Full of AI Dogshit原评论 ↗
引用hprotagonist · 2024-02

“Available: The Curse of Recursion”

贴出论文,解释为何用生成数据训练会让模型遗忘
You Sound Like a Bot原评论 ↗
引用almatabata · 2024-03

“There is the threat of model collapse though as discussed in this paper:”

提醒存在模型坍塌的风险,这篇论文对此有讨论
Generative AI and the widening software developer knowledge gap原评论 ↗
引用dangerwill · 2024-04

“Which studies show this? The Curse of Recursion shows the exact opposite”

反驳对方,称这篇论文显示的恰恰相反
Tell HN: We should snapshot a mostly AI output free version of the web原评论 ↗
引用vineyardlabs · 2024-05

“some cursory indication that in the long tail, training LLMs on LLM-generated data causes model collapse.”

引用它作为初步迹象:在长尾上用 LLM 生成数据训练 LLM 会导致模型坍塌
No "Zero-Shot" Without Exponential Data原评论 ↗
引用groby_b · 2024-06

“Model collapse itself is(was?) a fairly serious research topic: The Curse of Recursion”

指出模型坍塌是相当严肃的研究课题,并贴出论文
Gemma 2: Improving Open Language Models at a Practical Size [pdf]原评论 ↗
被提交为 story · 3 次