推荐claytonwramsey · 2023-07
“The paper is quite approachable and worth a quick scan.”
推荐快速浏览,易读,重复使用生成输出会得到怪异结果
用自身生成数据反复训练生成模型会导致质量或多样性衰退
论文用理论与实验分析了生成图像模型在「自噬循环」中的表现,即后一代模型用前几代模型生成的样本来训练,并比较了真实数据固定或新鲜、采样是否偏向质量等三类设置。核心结论是:如果每一代没有足够的新鲜真实数据,生成模型的精度(precision)或多样性(recall)会逐代下降,作者将这种现象称为模型自噬紊乱(MAD)。
HN 上的人用它说明拿 AI 生成内容训练 AI 会让模型退化,每一代都需要新鲜的真实数据。
“The paper is quite approachable and worth a quick scan.”
“Except you can't train AI on AI generated content.”
“This is a similar problem to what was observed in Diffusion models going”
“They need fresh real data every generation or they go downhill: Self-Consuming Generative Models Go MAD”