按年被提到
20242024 · 2 次2026
视觉、语音与图像生成›视觉语言与自监督表征·arXiv:2402.13217 · cs.CV

VideoPrism: A Foundational Visual Encoder for Video Understanding

Long Zhao, Nitesh B. Gundavarapu, Liangzhe Yuan, Hao Zhou, Shen Yan, Jennifer J. Sun, Luke Friedman, Rui Qian, Tobias Weyand, Yue Zhao, Rachel Hornung, Florian Schroff 等 19 人 · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2024-02-21 · 最近 2024-05-09
2不同的人
2活跃月份
2讨论串
34被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We introduce VideoPrism, a general-purpose video encoder that tackles diverse video understanding tasks with a single frozen model. We pretrain VideoPrism on a heterogeneous corpus containing 36M high-quality video-caption pairs and 582M video clips with noisy parallel text (e.g., ASR transcripts). The pretraining approach improves upon masked autoencoding by global-local distillation of semantic video embeddings and a token shuffling scheme, enabling VideoPrism to focus primarily on the video modality while leveraging the invaluable text associated with videos. We extensively test VideoPrism on four broad groups of video understanding tasks, from web video question answering to CV for science, achieving state-of-the-art performance on 31 out of 33 video understanding benchmarks. Our models are released at https://github.com/google-deepmind/videoprism.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论fngjdflmdflg · 2024-05

“An experiment being reproducible just means that if you repeat the experiment you will get the same outcome. We don't know if this experiment is reproducible or not because nobody has tried to reproduce it. I think if this paper gets citations, that will show that other people have read the paper[0] and gained something from it. So we can just wait and se…”

VideoPrism: A foundational visual encoder for video understanding原评论 ↗
被提交为 story · 1 次