按年被提到
20162016 · 4 次2026
视觉、语音与图像生成›卷积网络与图像识别·arXiv:1603.09382 · cs.LG

Deep Networks with Stochastic Depth

Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, Kilian Weinberger · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 4 人首次 2016-04-01 · 最近 2016-08-24
4不同的人
2活跃月份
4讨论串
2,127被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Very deep convolutional networks with hundreds of layers have led to significant reductions in error on competitive benchmarks. Although the unmatched expressiveness of the many layers can be highly desirable at test time, training very deep networks comes with its own set of challenges. The gradients can vanish, the forward flow often diminishes, and the training time can be painfully slow. To address these problems, we propose stochastic depth, a training procedure that enables the seemingly contradictory setup to train short networks and use deep networks at test time. We start with very deep networks but during training, for each mini-batch, randomly drop a subset of layers and bypass them with the identity function. This simple approach complements the recent success of residual networks. It reduces training time substantially and improves the test error significantly on almost all data sets that we used for evaluation. With stochastic depth we can increase the depth of residual networks even beyond 1200 layers and still yield meaningful improvements in test error (4.91% on CIFAR-10).

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论sja · 2016-08

“Good list! I think it's important to note that this article is (intentionally) focused on modern CNN architectures, and not "deep learning" in general. I'd also add in the following "technique" articles: Geoff Hinton et al.'s dropout paper[0] and Loffe and Szegedy's Batch Normalization paper[1]. I don't think there's been enough time for the dust to sett…”

Some of the most important deep learning papers原评论 ↗
被提交为 story · 3 次