“Super interesting! Immediately reminded of Iterative alpha-(de)Blending [1] which also sets out to set up a conceptually simpler diffusion model, and also arrives at formulating it as an approximate iterative projection process - I think this approach allows for more interesting experiments like the denoiser error analysis, though. [1] https://arxiv.org…”
Iterative $α$-(de)Blending: a Minimalist Deterministic Diffusion Model
We derive a minimalist but powerful deterministic denoising-diffusion model. While denoising diffusion has shown great success in many domains, its underlying theory remains largely inaccessible to non-expert users. Indeed, an understanding of graduate-level concepts such as Langevin dynamics or score matching appears to be required to grasp how it works. We propose an alternative approach that requires no more than undergrad calculus and probability. We consider two densities and observe what happens when random samples from these densities are blended (linearly interpolated). We show that iteratively blending and deblending samples produces random paths between the two densities that converge toward a deterministic mapping. This mapping can be evaluated with a neural network trained to deblend samples. We obtain a model that behaves like deterministic denoising diffusion: it iteratively maps samples from one density (e.g., Gaussian noise) to another (e.g., cat images). However, compared to the state-of-the-art alternative, our model is simpler to derive, simpler to implement, more numerically stable, achieves higher quality results in our experiments, and has interesting connections to computer graphics.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“For more info on why flow-matching is more stable than DDIM, see Heitz 2023, which nicely explains how they're almost equivalent but DDIM corresponds to a differential equation with a 1/α term that diverges at the start of the denoising process, while flow-matching/IADB doesn't diverge: https://arxiv.org/abs/2305.03486”