按年被提到
20122012 · 2 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1207.0580 · cs.NE

Improving neural networks by preventing co-adaptation of feature detectors

通过防止特征检测器的共适应来改进神经网络
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov · 2012

提出 dropout:训练时随机丢弃一半特征检测器

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 8 人首次 2012-11-04 · 最近 2023-07-26
8不同的人
8活跃月份
8讨论串
6,380被引(OpenAlex)

大型前馈网络在小训练集上训练时容易过拟合;在每个训练样本上随机省略一半特征检测器可大幅减轻过拟合,因为这阻止了复杂的共适应:每个神经元不再只在某几个特定检测器的配合下才有用,而是学到在组合上大量不同的内部情境中都通用的特征。这种随机 dropout 在许多基准任务上带来很大提升,并刷新了语音和物体识别的纪录。

HN 为什么提它

它被当作 dropout 的出处和典型的神经网络正则化技术来引用,评论者借它解释共适应、噪声注入,以及与遗传重组带来鲁棒性的类比。

奠基之作难度 3 / 5约 20 分钟经得起时间
读之前最好知道
1前馈神经网络与反向传播
2过拟合与正则化
3集成学习的直觉

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
推荐sja · 2016-08

“I'd also add in the following "technique" articles: Geoff Hinton et al.'s dropout paper[0]”

推荐 Hinton 等的 dropout 论文,归入技术类文章
Some of the most important deep learning papers原评论 ↗
引用psb217 · 2012-12

“You can read about it in their paper”

引它介绍 dropout 如何防止特征共适应
Why use SVM原评论 ↗
引用thisisdave · 2014-02

“The best-known example is [0], but he's also had some success just adding noise to the calculation of the gradient”

举 dropout 为向神经网络训练注入噪声的最知名例子
Computing 10,000X more efficiently (2010) [pdf]原评论 ↗
引用billderose · 2014-05

“Hinton's DropOut [1] and Wan's DropConnect [2] have ameliorated some of the overfitting issues present in traditional NN's.”

提到 Hinton 的 DropOut 缓解了传统神经网络的过拟合
What does a neural network actually do?原评论 ↗
引用eivarv · 2015-10

“very popular until dropout [4,5] (and its variations) appeared.”

指出 dropout 出现后,取代了无监督预训练的流行地位
Theoretical Motivations for Deep Learning原评论 ↗
引用spacehacker · 2016-11

“The dropout regularization method for neural networks was sort of inspired by this observation, i.e. that recombination (combining genes of more than one individual) makes the resulting structures more robust [1].”

指出 dropout 的灵感来自重组让结构更稳健的观察
Learning about evolution from species who thrive without sexual reproduction原评论 ↗
看全部 7 条 →
引用esafak · 2023-07

“Co-adaptation reduction is another, and it is an intuitive one”

引它说明 dropout 减少共适应,让神经元在多种上下文中预测
What We Know About LLMs (A Primer)原评论 ↗
被提交为 story · 1 次