按年被提到
20222023 · 2 次2026
视觉、语音与图像生成›视觉语言与自监督表征·arXiv:2209.06794 · cs.CV

PaLI: A Jointly-Scaled Multilingual Language-Image Model

Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver 等 29 人 · 2022
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2023-05-04 · 最近 2024-02-22
3不同的人
2活跃月份
3讨论串
391被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Effective scaling and a flexible task interface enable large language models to excel at many tasks. We present PaLI (Pathways Language and Image model), a model that extends this approach to the joint modeling of language and vision. PaLI generates text based on visual and textual inputs, and with this interface performs many vision, language, and multimodal tasks, in many languages. To train PaLI, we make use of large pre-trained encoder-decoder language models and Vision Transformers (ViTs). This allows us to capitalize on their existing capabilities and leverage the substantial cost of training them. We find that joint scaling of the vision and language components is important. Since existing Transformers for language are much larger than their vision counterparts, we train a large, 4-billion parameter ViT (ViT-e) to quantify the benefits from even larger-capacity vision models. To train PaLI, we create a large multilingual mix of pretraining tasks, based on a new image-text training set containing 10B images and texts in over 100 languages. PaLI achieves state-of-the-art in multiple vision and language tasks (such as captioning, visual question-answering, scene-text understanding), while retaining a simple, modular, and scalable design.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论runnerup · 2023-05

“http://www.incompleteideas.net/IncIdeas/BitterLesson.html Generally we have continued finding that the more "other"/general stuff an AI model is trained on, the better it performs on specific tasks. As in, an AI model trained to identify photos of all animals will perform better than an AI model that is only trained to identify breeds of dogs. Even at id…”

Replit's new Code LLM: Open Source, 77% smaller than Codex, trained in 1 week原评论 ↗
评论reaperman · 2023-05

“This model is vision-only so it can't be SOTA even if it's #1 performing in many of the original categories of benchmarks, which it is (it's a very very good model). We've moved on from ImageNet-style tests "Choose the most appropriate label for this image from 200 possible labels" to much more advanced "Reasoning" tests[0]. PaLI[1] is potentially the So…”

Meta Open-Sources Computer Vision Foundation Model DINOv2原评论 ↗
评论famouswaffles · 2024-02

“This explanation is wrong as I've already said (256 is not the result of any conversion to text) but no one has to take my word for it. From the Gemini report https://arxiv.org/abs/2312.11805 >The visual encoding of Gemini models is inspired by our own foundational work on Flamingo (Alayrac et al., 2022), CoCa (Yu et al., 2022a), and PaLI (Chen et al.,2…”

The killer app of Gemini Pro 1.5 is using video as an input原评论 ↗