按年被提到
20232023 · 2 次2026
视觉、语音与图像生成›视觉语言与自监督表征·arXiv:2307.12698 · cs.CV

MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features

Adrien Bardes, Jean Ponce, Yann LeCun · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-07-27 · 最近 2023-09-25
2不同的人
2活跃月份
2讨论串
13被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Self-supervised learning of visual representations has been focusing on learning content features, which do not capture object motion or location, and focus on identifying and differentiating objects in images and videos. On the other hand, optical flow estimation is a task that does not involve understanding the content of the images on which it is estimated. We unify the two approaches and introduce MC-JEPA, a joint-embedding predictive architecture and self-supervised learning approach to jointly learn optical flow and content features within a shared encoder, demonstrating that the two associated objectives; the optical flow estimation objective and the self-supervised learning objective; benefit from each other and thus learn content features that incorporate motion information. The proposed approach achieves performance on-par with existing unsupervised optical flow benchmarks, as well as with common self-supervised learning approaches on downstream tasks such as semantic segmentation of images and videos.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论jamesblonde · 2023-09

“You are correct that we have two modalities projecting into a shared latent space. However, that doesn't convey anything about how to jointly train the embedding models, does it? That's what the two tower embedding model shows - how to jointly train the models using distance functions like cosine distance. On an aside, others, as esteemed as Yann Lecun, …”

Two-Tower Embedding Model原评论 ↗
被提交为 story · 1 次