按年被提到
20232023 · 2 次2026
视觉、语音与图像生成›扩散模型·arXiv:2306.05720 · cs.CV

Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model

Yida Chen, Fernanda Viégas, Martin Wattenberg · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-08-21 · 最近 2024-01-26
2不同的人
3活跃月份
3讨论串
8被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Latent diffusion models (LDMs) exhibit an impressive ability to produce realistic images, yet the inner workings of these models remain mysterious. Even when trained purely on images without explicit depth information, they typically output coherent pictures of 3D scenes. In this work, we investigate a basic interpretability question: does an LDM create and use an internal representation of simple scene geometry? Using linear probes, we find evidence that the internal activations of the LDM encode linear representations of both 3D depth data and a salient-object / background distinction. These representations appear surprisingly early in the denoising process$-$well before a human can easily make sense of the noisy images. Intervention experiments further indicate these representations play a causal role in image synthesis, and may be used for simple high-level editing of an LDM's output. Project page: https://yc015.github.io/scene-representation-diffusion-model/

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论l33tman · 2023-11

“Certainly: https://arxiv.org/abs/2306.05720 It's a very interesting paper. "Even when trained purely on images without explicit depth information, they typically output coherent pictures of 3D scenes. In this work, we investigate a basic interpretability question: does an LDM create and use an internal representation of simple scene geometry? Using line…”

Show HN: Dalle-3 and GPT4-Vision feedback loop原评论 ↗
评论l33tman · 2024-01

“Please see https://arxiv.org/abs/2306.05720 Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model "... In this work, we investigate a basic interpretability question: does an LDM create and use an internal representation of simple scene geometry? Using linear probes, we find evidence that the internal activations of the LDM encode…”

Lumiere: A space-time diffusion model for realistic video generation原评论 ↗
被提交为 story · 1 次