按年被提到
20242025 · 1 次2026
视觉、语音与图像生成›扩散模型·arXiv:2406.11838 · cs.CV

Autoregressive Image Generation without Vector Quantization

Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, Kaiming He · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2025-03-26 · 最近 2026-09-17
2不同的人
2活跃月份
2讨论串
82被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Conventional wisdom holds that autoregressive models for image generation are typically accompanied by vector-quantized tokens. We observe that while a discrete-valued space can facilitate representing a categorical distribution, it is not a necessity for autoregressive modeling. In this work, we propose to model the per-token probability distribution using a diffusion procedure, which allows us to apply autoregressive models in a continuous-valued space. Rather than using categorical cross-entropy loss, we define a Diffusion Loss function to model the per-token probability. This approach eliminates the need for discrete-valued tokenizers. We evaluate its effectiveness across a wide range of cases, including standard autoregressive models and generalized masked autoregressive (MAR) variants. By removing vector quantization, our image generator achieves strong results while enjoying the speed advantage of sequence modeling. We hope this work will motivate the use of autoregressive generation in other continuous-valued domains and applications. Code is available at: https://github.com/LTH14/mar.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论zaptrem · 2025-03

“Token by token diffusion was done by MAR https://arxiv.org/abs/2406.11838 and Fluid (scaled up MAR) https://arxiv.org/abs/2410.13863 When you randomly pick the locations they found it worked okay, but doing it in raster order (left to right, top to bottom) they found it didn't work as well. We tried it for music and found it was vulnerable to compounding…”

4o Image Generation原评论 ↗
评论schopra909 · 2026-09

“Just skimmed the paper (haven't seen this before). Might need to read more carefully, but at first glance I don't understand their intuition why the "Markovian property limits the model’s ability to fully utilize the generation trajectory". When it comes to multi-resolution training (e.g. matryoshka training), there are precedents that don't require this…”

Training Text-to-Image Models 3.6× Faster原评论 ↗