“you need these tools to understand Stable Diffusion which really is just Latent Diffusion[4]”
High-Resolution Image Synthesis with Latent Diffusion Models
在自编码器潜空间里做扩散,Stable Diffusion 的原论文
论文不在像素空间而在预训练自编码器的潜空间中训练扩散模型,在复杂度降低与细节保留之间取得良好折中,大幅减少训练和推理的算力需求。通过引入交叉注意力层,模型可接受文本、边界框等通用条件输入,并能以卷积方式做高分辨率合成。潜在扩散模型在图像修复上达到当时最优,在无条件生成、语义场景合成和超分辨率上也具有竞争力。
HN 上的人把它当作 Stable Diffusion 背后的原始论文:说明“在潜空间做扩散”如何大幅降低算力,从而让图像生成走向大众。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“Many of you may have experienced this when the first stable diffusion paper came out and the images in the paper were incredible but when you used the hugging face generator you'd get nothing nearly as good.”
“They continued building off their latent diffusion direction (encode with vqgan-vae and then diffusion in latent space)”
“after that Latent Diffusion Models [4] democratized the thing by vastly reducing the amount of computation needed - operating in latent space”
“another reason for the small model size and the novelty of the underlying paper [1], is that the diffusion model is not acting on the pixel space but rather on a latent space.”
“Very interesting, although I was expecting the Stable Diffusion paper[1] to be one of the most cited in 2022”
看全部 9 条 →
“Links to High-Resolution Image Synthesis with Latent Diffusion Models which is the Version 2 of the paper, indeed released in April 2022.”
“This is explained in the original latent diffusion paper on which Stable Diffusion was based: High-Resolution Image Synthesis with Latent Diffusion Models”
“I only implemented the initial SD paper [1] back then so I have no clue whether the u-net architecture changed when it comes to the semantic mapping of the text/labels.”