引用edouard-harris · 2022-05
“The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].)”
以 Flamingo 为例,说明儿童图画书理解问题已被解决
桥接预训练视觉与语言模型、可少样本适配的视觉语言模型
Flamingo 是一类视觉语言模型(VLM),通过架构创新桥接强大的纯视觉和纯语言预训练模型,处理任意交错的图文序列,并无缝接收图像或视频输入。它在含交错图文的大规模网络语料上训练,从而具备上下文少样本学习能力;仅靠提示几个任务示例,单个模型就在许多图像和视频任务上取得新的最好成绩,在多个基准上超过用成千倍任务数据微调的模型。
评论者把它当作图文交错序列理解的代表模型,分别引用于“儿童图画书问题已解决”、Gemini 的视觉编码和 OpenTSLM 的基础模型。
“The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].)”
“Flamingo - Flamingo”
“This is a system for understanding interleaved text and images in sequence.”