“For those who don't know, that is the idea behind ResNet (He et al., Deep Residual Learning for Image Recognition, Deep Residual Learning for Image Recognition), one of the most influential papers in deep learning of all time.”
Deep Residual Learning for Image Recognition
用残差连接训练极深网络,赢得 ILSVRC 2015 冠军
论文提出残差学习框架:让网络层学习相对于层输入的残差函数,而不是未参照输入的函数,从而使远比以往更深的网络更容易优化,并能随深度增加提高精度。在 ImageNet 上,深达 152 层的残差网络比 VGG 深 8 倍但复杂度更低,集成模型的测试错误率为 3.57%,赢得 ILSVRC 2015 分类任务第一;同时在 CIFAR-10 上分析了 100 层和 1000 层网络,并在 COCO 检测上相对提升 28%。
HN 上的人多把 ResNet 当作深度学习史上最有影响力的论文之一,用来说明残差(跳跃)连接让极深网络可训练,并指出 Transformer 里的残差块和残差流思想都源自它。
学习路径
读这篇之前,按顺序读书架上的 6 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出这条路径从 ImageNet 基准出发,经过 VGG、Inception 的深度之争,以及批归一化、He 初始化、Highway 网络,建立理解残差学习为何出现的背景。
- 1
ResNet 的评测都在 ImageNet 上,先理解该基准与任务设置。
读抓 ILSVRC 的任务定义与评价指标;历史部分略读。
- 2
VGG 展示堆叠 3x3 卷积加深网络的收益,ResNet 的基础结构沿用了它。
读抓网络配置表和深度对精度的影响。
- 3
- 4
批归一化让更深的网络可以训练,ResNet 在每个卷积后使用它。
读抓对每个小批量归一化的公式及其在训练中的作用。
- 5
He 初始化和 PReLU 是训练很深网络的初始化基础,目标论文沿用了这套初始化。
读抓针对 ReLU 的权重初始化推导的想法;PReLU 部分略读。
- 6
- 7Deep Residual Learning for Image Recognition
提出残差学习,赢得 ILSVRC 2015 冠军。
读抓退化问题的实验、残差块的形式,以及 34 层与 152 层的结构。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“These are a few of the interesting, for me personally, since then:”
“Citation 4000+. Milestone of image recognition.”
“The foundation for many of the current SOA papers is He, et. al:”
“I think that's a pretty good thing they produced.”
“here's a direct link to the paper on arXiv: Deep Residual Learning for Image Recognition”
看全部 20 条 →
“Deep Residual Learning for Image Recognition Deep Residual Learning for Image Recognition”
“specifically deep residual learning”
“which introduces identity functions into the layer equation”
“higher level building blocks like LSTMs[1], gated recurrent units[2], highway layers[3], batch normalization[4], and residual blocks[5]”
“Deep Residual Learning for Image Recognition”
“One of the most successful recent innovations is the ResNet architectures, and the related highway networks.”
“Transfomers use residual blocks, which were introduced by the ResNet CNN arhitecture (2015, Deep Residual Learning for Image Recognition)”
“How similar is this to creating a resnet? [1].”
“Haven't read thoroughly but it seems they are investigating ResNet models”
“Then you get ResNets[1] with skip connections and move forward to today.”
“ViT (ICLR 21) and DeiT (ICML 21)[4,5], and even ResNet (CVPR 16)[6] and tons of ReLU papers we can list.”
“Deep Residual Learning for Image Recognition (ResNet): Deep Residual Learning for Image Recognition”
“Deep Residual Learning for Image Recognition - Deep Residual Learning for Image Recognition”
“each layer adds the ability to provide more information to the residual stream [1], [2]”