按年被提到
20232023 · 1 次2026
大模型与语言模型›对话式大模型·arXiv:2303.04671 · cs.CV

Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models

Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, Nan Duan · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-03-11 · 最近 2024-06-06
2不同的人
2活跃月份
2讨论串
430被引(OpenAlex)
arXiv 摘要(候选层不写导读)

ChatGPT is attracting a cross-field interest as it provides a language interface with remarkable conversational competency and reasoning capabilities across many domains. However, since ChatGPT is trained with languages, it is currently not capable of processing or generating images from the visual world. At the same time, Visual Foundation Models, such as Visual Transformers or Stable Diffusion, although showing great visual understanding and generation capabilities, they are only experts on specific tasks with one-round fixed inputs and outputs. To this end, We build a system called \textbf{Visual ChatGPT}, incorporating different Visual Foundation Models, to enable the user to interact with ChatGPT by 1) sending and receiving not only languages but also images 2) providing complex visual questions or visual editing instructions that require the collaboration of multiple AI models with multi-steps. 3) providing feedback and asking for corrected results. We design a series of prompts to inject the visual model information into ChatGPT, considering models of multiple inputs/outputs and models that require visual feedback. Experiments show that Visual ChatGPT opens the door to investigating the visual roles of ChatGPT with the help of Visual Foundation Models. Our system is publicly available at \url{https://github.com/microsoft/visual-chatgpt}.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论aleksanderhan · 2024-06

“Turbo-Genius just got image capabilities. Still early days, but I'm planning on implementing this paper: https://arxiv.org/abs/2303.04671 - only with local models.”

Show HN: Turbo-Genius原评论 ↗
被提交为 story · 1 次