按年被提到
20222022 · 1 次2026
视觉、语音与图像生成›图像与视频生成·arXiv:2204.06125 · cs.CV

Hierarchical Text-Conditional Image Generation with CLIP Latents

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, Mark Chen · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-05-02 · 最近 2024-03-16
2不同的人
2活跃月份
2讨论串
4,216被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We show that explicitly generating image representations improves image diversity with minimal loss in photorealism and caption similarity. Our decoders conditioned on image representations can also produce variations of an image that preserve both its semantics and style, while varying the non-essential details absent from the image representation. Moreover, the joint embedding space of CLIP enables language-guided image manipulations in a zero-shot fashion. We use diffusion models for the decoder and experiment with both autoregressive and diffusion models for the prior, finding that the latter are computationally more efficient and produce higher-quality samples.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论benlivengood · 2022-05

“The original paper examines some likely causes for failure to encode relational details between subjects, e.g. the corgi with a red hat and green necktie or vice versa chosen seemingly at random. The model seems to learn about contrasting subjects but not track particularly which attributes belong to which subjects. This is, as far as I can tell from the …”

A preliminary analysis of DALL-E 2 (Marcus, Davis, Aaronson)原评论 ↗
评论gwern · 2024-03

“Correct. Most image models to date have used very small text models, which are unable to spell many words (https://arxiv.org/abs/2108.11193) - if you cannot spell a word letter by letter because you don't know what the letters are, how are you going to generate a pixel image of each of those letters...? (It is not as if letters were intrinsically hard. GA…”

Ask HN: Why can't image generation models spell?原评论 ↗