“> It doesn’t go out and google for reference material based on the prompt. And note that even if it did (which is definitely architecturally possible given progress in retrieval and demos like WebGPT), it still wouldn't necessarily be 'copying', in the goalpost-moving excuse. Facebook has a neat diffusion model which uses retrieval on a dataset of image…”
KNN-Diffusion: Image Generation via Large-Scale Retrieval
Recent text-to-image models have achieved impressive results. However, since they require large-scale datasets of text-image pairs, it is impractical to train them on new domains where data is scarce or not labeled. In this work, we propose using large-scale retrieval methods, in particular, efficient k-Nearest-Neighbors (kNN), which offers novel capabilities: (1) training a substantially small and efficient text-to-image diffusion model without any text, (2) generating out-of-distribution images by simply swapping the retrieval database at inference time, and (3) performing text-driven local semantic manipulations while preserving object identity. To demonstrate the robustness of our method, we apply our kNN approach on two state-of-the-art diffusion backbones, and show results on several different datasets. As evaluated by human studies and automatic metrics, our method achieves state-of-the-art results compared to existing approaches that train text-to-image generation models using images only (without paired text data)
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“> The AI does NOT build or use reference boards for specific prompts. Interestingly, even if it did build reference boards, that doesn't mean that the generated image would 'copy' or 'look like' the references all that much. We know this because there are diffusion generative models which do exactly that: they do retrievals first, and include the 'exampl…”
- KNN-Diffusion: Image Generation via Large-Scale Retrieval ↗lnyan · 2022-10-24 · 2 分