按年被提到
20172017 · 1 次2026
视觉、语音与图像生成›视觉语言与自监督表征·arXiv:1704.07804 · cs.CV

SfM-Net: Learning of Structure and Motion from Video

Sudheendra Vijayanarasimhan, Susanna Ricco, Cordelia Schmid, Rahul Sukthankar, Katerina Fragkiadaki · 2017
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2017-04-28 · 最近 2019-11-05
2不同的人
2活跃月份
2讨论串
327被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We propose SfM-Net, a geometry-aware neural network for motion estimation in videos that decomposes frame-to-frame pixel motion in terms of scene and object depth, camera motion and 3D object rotations and translations. Given a sequence of frames, SfM-Net predicts depth, segmentation, camera and rigid object motions, converts those into a dense frame-to-frame motion field (optical flow), differentiably warps frames in time to match pixels and back-propagates. The model can be trained with various degrees of supervision: 1) self-supervised by the re-projection photometric error (completely unsupervised), 2) supervised by ego-motion (camera motion), or 3) supervised by depth (e.g., as provided by RGBD sensors). SfM-Net extracts meaningful depth estimates and successfully estimates frame-to-frame camera rotations and translations. It often successfully segments the moving objects in the scene, even though such supervision is never provided.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论osdf · 2017-04

“There seem to be some nice developments into this direction recently? Something like the SfMNet (https://arxiv.org/abs/1704.07804) or the Correlation Filter Net (http://www.robots.ox.ac.uk/~luca/cfnet.html) for tracking.”

Have We Forgotten about Geometry in Computer Vision?原评论 ↗
评论Mathnerd314 · 2019-11

“It seems to do motion transfer alright but the legs in the dancing video get warped instead of moving so there's some frames where they have triangle legs vanishing at the knee. https://youtu.be/8AZBuyEuDqc?t=34 Maybe if you scale up the training data the model is going to learn some better warping, but I think to really get better photorealism there has…”

Few-Shot Video-to-Video Synthesis原评论 ↗