“There are some survey papers like this: Deep Learning in Neural Networks”
Deep Learning in Neural Networks: An Overview
Schmidhuber 对深度学习及其前史的长篇历史综述
这篇历史性综述简明地总结了深度神经网络(包括循环网络)在模式识别和机器学习中的相关工作,其中许多出自上个千年。作者以信用分配路径(credit assignment path)的深度区分浅层与深层学习器,并回顾深度监督学习(含反向传播的历史)、无监督学习、强化学习与进化计算,以及对编码大型深度网络的短程序的间接搜索。
HN 上的人把它当作深度学习历史和文献的全景式参考,称它是篇幅很长的首选综述,引用很全,不过也有人提到它源于对前人贡献被忽略的不满。
学习路径
读这篇之前,按顺序读书架上的 5 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出这条路径选取综述中提到的几项近期代表性成果:无监督特征学习、dropout、卷积网络可视化、深度强化学习,帮助把历史综述与当时的实证进展对上号。
- 1
大规模无监督学习让网络自发学出高层特征,对应综述中无监督深度学习的部分。
读抓网络结构与学出人脸检测器的实验结果。
- 2
dropout 是综述所述正则化的重要进展。
读抓随机丢弃一半特征检测器的做法及其效果。
- 3
可视化卷积网络各层学到的特征,帮助理解综述中的深度监督学习。
读抓反卷积式可视化展示逐层特征的结果。
- 4
- 5
同一作者的通用问题求解器思路,对应综述中关于通用学习与间接搜索的论述。
读抓不断寻找最简单的未解问题这一想法;细节略读。
- 6Deep Learning in Neural Networks: An Overview
Schmidhuber 对深度学习及其前史的长篇历史综述。
读抓信用分配路径深度的概念、反向传播的历史与各类学习范式的脉络。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“the above was recommended by deeplearning.net and is what I'm slogging through ATM.”
“For more details, refer to Schmidhuber's own website: [LINK] and his excellent review paper”
“here is an excellent historical survey paper:”
“this other one's the go-to article (though it's much longer): Deep Learning in Neural Networks”
“there are quite a few lists out there if you look.”
看全部 10 条 →
“He wrote an enormous literature review of deep learning [0] basically because he felt that people weren't crediting ideas enough.”
“For a history of the subject, see Section 5.1 of [2].”
“I think Schmidhuber introduced that definition many years ago, Deep Learning in Neural Networks”
“Conveniently skim-read over the course of the four weekends on one month.”
- Deep learning in neural networks: An overview ↗dave_sullivan · 2014-05-02 · 1 分
- Deep Learning in Neural Networks: An Overview ↗kaycebasques · 2023-12-18 · 1 分