“This paper is a decent starting point on the literature side, but it's a doozy.”
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
最早的扩散模型:先逐步加噪破坏数据结构,再学习反向去噪过程
为了同时获得灵活性与可处理性,作者受非平衡统计物理启发:通过迭代的前向扩散过程系统而缓慢地破坏数据分布中的结构,然后学习一个恢复结构的反向扩散过程,得到灵活且可处理的数据生成模型。这使得在有数千层或时间步的深度生成模型中也能快速学习、采样和评估概率,并可计算学到模型下的条件概率和后验概率;论文同时发布了开源参考实现。
HN 上的人把它当作扩散模型的源头来引用,称它在 2015 年就提出了生成式扩散模型,并把它当作学习扩散模型文献的起点(但数学较重)。
学习路径
读这篇之前,按顺序读书架上的 3 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先理解变分自编码器和可逆流模型这两类生成模型,以及评估生成模型的难点,再读扩散模型的最初形式。
- 1
- 2
NICE 是同期另一种精确似然的生成模型,可与扩散模型的可计算性目标对照。
读可逆变换与精确对数似然。
- 3
- 4Deep Unsupervised Learning using Nonequilibrium Thermodynamics
目标论文:用逐步加噪的前向扩散与学到的反向去噪过程构造灵活且可处理的生成模型。
读前向扩散核、反向过程的参数化、训练目标(对数似然下界),以及去噪与图像修复应用。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Your goal should be to understand this paper[2].”
“The real reason is that big steps result in worse results (1);”
“The core idea of a Markov chain[0] where each state change leads to higher likelihood, is bound to work, even if the rest doesn’t.”
“The predecessor to DDPM (the work that kicked off the diffusion model era) was published in 2015[0], only a year after GANs[1].”
“The original Dickstein 2015 paper [1] formulated diffusion as maximizing (a lower bound of) log-likelihood of generating the distribution, so there was some theory.”
看全部 9 条 →
“[6] Deep Unsupervised Learning using Nonequilibrium Thermodynamics”
“There is an analogy between random noise -> image and a random distribution of gas particles -> concentrated volume of particles.”
“Invented the first generative diffusion model in 2015. Deep Unsupervised Learning using Nonequilibrium Thermodynamics”
- Deep Unsupervised Learning Using Nonequilibrium Thermodynamics [pdf] ↗psoto · 2015-05-23 · 1 分