“Learning motor torques directly from vision is a very important result.”
End-to-End Training of Deep Visuomotor Policies
用 CNN 直接把原始图像映射为机器人电机力矩,端到端训练
论文探讨感知与控制联合端到端训练是否优于分别训练。作者开发了可学习把原始图像观测直接映射为机器人电机力矩的策略方法:策略用仅 92,000 个参数的深度 CNN 表示,通过部分可观测的引导策略搜索训练,该方法把策略搜索转化为监督学习,监督来自简单的轨迹中心强化学习方法。作者在需要视觉与控制紧密配合的真实操作任务(如把瓶盖拧到瓶子上)上评估,并与一系列已有策略搜索方法做仿真对比。
HN 上的人称赞「直接从视觉学习电机力矩」是重要成果,并把它作为机器人端到端学习的起点,后续新工作可顺着其引用追查。
学习路径
读这篇之前,按顺序读书架上的 2 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出先熟悉深度 RL 从像素学习的设定,再了解大型 CNN 的预训练,最后读目标论文如何把感知与控制联合端到端训练。
- 1
它展示卷积网络直接从像素学习控制,是目标论文设定的参照。
读Q 学习加卷积网络的做法;对比目标论文用策略搜索处理连续力矩。
- 2
目标论文的视觉层可用预训练的大规模图像分类 CNN 初始化,VGG 是代表性的网络。
读小卷积核堆叠的结构;不必看 ImageNet 结果表。
- 3End-to-End Training of Deep Visuomotor Policies
目标论文:用 CNN 把原始图像直接映射到电机力矩,联合训练感知与控制。
读引导式策略搜索(guided policy search)、空间 softmax 特征点层,以及端到端与分开训练的对比。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“this paper initially demonstrated an impressive improvement in manipulative tasks, and you can prob follow its numerous citations for newer stuff:”
“Interestingly, they initialise the visual learning model using the ImageNet images.”
“but which requires too much computation to actually apply directly at test time, as in End-to-End Training of Deep Visuomotor Policies”
“Policy search methods assume that there aren't many trajectories that make sense”
“[3] End-to-End Training of Deep Visuomotor Policies”
看全部 8 条 →
“Michie's nascent hybrid of raw sensor machine learning & GOFAI embodied in robots is just being rediscovered in Abeel & Levines work today. End-to-End Training of Deep Visuomotor Policies”
“Abeel and Levine's work[2] where robots learn robust functions to map from pixels to motor torques for domestic tasks show solutions built on features learnt from raw data are more resilient than hand-engineered ones.”
- End-to-End Training of Deep Visuomotor Policies [pdf] ↗evc123 · 2015-05-26 · 5 分
- End-To-End Training of Deep Visuomotor Policies ↗jcr · 2015-07-07 · 6 分