“A fairly new but promising approach for autoregression that seems to scale as well as diffusion is predicting the next image scale/resolution rather than the next image patch.”
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
VAR:把图像自回归改为由粗到细的下一尺度预测,超过扩散 Transformer
论文提出视觉自回归建模(VAR),把图像上的自回归学习重新定义为由粗到细的“下一尺度预测”,不再是标准的光栅扫描式“下一 token 预测”。这种简单直观的做法让自回归 Transformer 能快速学习视觉分布并泛化良好,首次使类 GPT 自回归模型在图像生成上超过扩散 Transformer。在 ImageNet 256x256 上,VAR 把 FID 从 18.65 降到 1.73,IS 从 80.4 升到 350.2,推理速度约快 20 倍,并在图像质量、速度、数据效率和可扩展性等方面优于 DiT。扩大 VAR 模型呈现类似 LLM 的幂律扩展,并具备图像修复、扩图和编辑等零样本泛化能力。
HN 上的人把它当作比扩散模型扩展性相当甚至更好的新型自回归图像生成方法,并猜测一些新的图像模型可能采用了类似 VAR 的由粗到细思路。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“But recently (2024 NeuIPS paper of the year) there was a new paper on autoregressive image modelling that apparently outperforms diffusion models: Visual Autoregressive Modeling”
“There is a more recent approach to auto-regressive image generation.”
“It's probably an implementation of VAR”
“i think it's more likely an implementation of VAR - Visual Autoregressive Modeling.”
“This is very similar to VAR [1]”
看全部 8 条 →
- Visual Autoregressive Modeling: Scalable Image Generation W NextScale Prediction ↗pksebben · 2024-04-05 · 2 分
- Visual Autoregressive Modeling: Image Generation via Next-Resolution Prediction ↗famouswaffles · 2024-10-05 · 1 分