“The second is model collapse which is essentially just bad data making its way into the training set, once again a sufficiently curated corpus eliminates this problem.”
The Curse of Recursion: Training on Generated Data Makes Models Forget
用模型生成的内容训练,会造成不可逆的模型坍缩
论文考察当大语言模型生成了网上大部分文本之后,GPT-{n} 会怎样。作者发现,在训练中使用模型生成的内容会让所得模型出现不可逆的缺陷,原始内容分布的尾部会消失,他们称之为“模型坍缩(Model Collapse)”,并表明它可出现在变分自编码器、高斯混合模型和 LLM 中。论文给出了理论直觉,指出这是各类习得的生成模型共有的现象,并认为在 LLM 内容充斥网络的背景下,真实人类交互数据将更加宝贵。
评论者拿它说明互联网被 AI 生成内容污染后训练数据会退化(模型坍缩),也有人反驳:论文用的是不做任何筛选的玩具设定,精心整理的语料可以避免这个问题。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“had to make do with very different models as proxies”
“are totally inapplicable to the real world, as opposed to their artificial toy model where no kind of curation or selection whatsoever is done”
“Basically, if this paper is right, 2021 was the last year usable training data could be pulled from the internet.”
“There are first studies on this topic”
“it leads to complete gibberish”
看全部 14 条 →
“there is a model-collapse phenomena”
“This process is called "neural network collapse". The Curse of Recursion”
“in his article The Curse of Recursion is already considering a scenario where most of the Internet will be a neural network-generated hallucination.”
“Available: The Curse of Recursion”
“There is the threat of model collapse though as discussed in this paper:”
“Which studies show this? The Curse of Recursion shows the exact opposite”
“some cursory indication that in the long tail, training LLMs on LLM-generated data causes model collapse.”
“Model collapse itself is(was?) a fairly serious research topic: The Curse of Recursion”
- The Curse of Recursion: Training on Generated Data Makes Models Forget ↗YeGoblynQueenne · 2023-06-03 · 2 分
- The Curse of Recursion: Training on Generated Data Makes Models Forget ↗indus · 2023-06-13 · 1 分
- The Curse of Recursion: Training on generated data makes models forget (2023) ↗surprisetalk · 2024-12-01 · 122 分