“I'd also add in the following "technique" articles: Geoff Hinton et al.'s dropout paper[0]”
Improving neural networks by preventing co-adaptation of feature detectors
提出 dropout:训练时随机丢弃一半特征检测器
大型前馈网络在小训练集上训练时容易过拟合;在每个训练样本上随机省略一半特征检测器可大幅减轻过拟合,因为这阻止了复杂的共适应:每个神经元不再只在某几个特定检测器的配合下才有用,而是学到在组合上大量不同的内部情境中都通用的特征。这种随机 dropout 在许多基准任务上带来很大提升,并刷新了语音和物体识别的纪录。
它被当作 dropout 的出处和典型的神经网络正则化技术来引用,评论者借它解释共适应、噪声注入,以及与遗传重组带来鲁棒性的类比。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“You can read about it in their paper”
“The best-known example is [0], but he's also had some success just adding noise to the calculation of the gradient”
“Hinton's DropOut [1] and Wan's DropConnect [2] have ameliorated some of the overfitting issues present in traditional NN's.”
“very popular until dropout [4,5] (and its variations) appeared.”
“The dropout regularization method for neural networks was sort of inspired by this observation, i.e. that recombination (combining genes of more than one individual) makes the resulting structures more robust [1].”
看全部 7 条 →
“Co-adaptation reduction is another, and it is an intuitive one”
- Improving neural networks by preventing co-adaptation of feature detectors [pdf] ↗Mizza · 2012-11-04 · 2 分