按年被提到
20182020 · 8 次2026
机器学习与深度学习›深度学习理论与泛化·arXiv:1803.03635 · cs.LG

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

彩票假设:寻找稀疏且可训练的神经网络
Jonathan Frankle, Michael Carbin · 2018

稠密网络中藏着能单独训练到同等精度的稀疏子网络

开始阅读 ↗PDFalphaXiv 讨论
被推荐 · 30 人首次 2018-03-19 · 最近 2026-04-24
30不同的人
24活跃月份
31讨论串
1,529被引(OpenAlex)

论文发现,标准剪枝方法会自然地找出一些子网络,这些子网络因其初始化而能被有效训练,据此提出彩票假设:稠密、随机初始化的前馈网络含有「中奖彩票」子网络,单独训练时能在相近迭代次数内达到与原网络相当的测试精度。作者给出识别中奖彩票的算法和支持实验:在 MNIST 和 CIFAR10 的多种全连接与卷积网络上,找到的子网络可小于原网络的 10–20%,在此规模以上,它们学得更快、测试精度更高。

HN 为什么提它

HN 上的人常用它解释为什么过参数化网络能训练成功、为什么大部分权重可以去掉,有人称之为最有意思的关于神经网络为何有效的假设,也有人指出它并未提高样本效率。

方法难度 3 / 5约 60 分钟经得起时间
读之前最好知道
1神经网络训练与初始化
2网络剪枝(pruning)
3过参数化(overparameterization)与泛化
4MNIST、CIFAR10 等基础数据集与实验设置

学习路径

读这篇之前,按顺序读书架上的 4 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出

这条路径先建立 CNN 基准架构、模型压缩与剪枝的背景,再理解泛化为何不依赖网络大小,最后读彩票假设。

  1. 1
    Very Deep Convolutional Networks for Large-Scale Image Recognition面向大规模图像识别的极深卷积网络 · 2014

    VGG 是论文实验使用的卷积网络之一,其结构与训练设置是实验的基础。

    读抓 3x3 卷积堆叠的结构与网络配置;训练细节了解即可。

  2. 2
    Deep Residual Learning for Image Recognition用于图像识别的深度残差学习 · 2015

    ResNet 是论文在更深网络上做剪枝实验的架构,也要理解残差结构对训练的影响。

    读抓残差块与 shortcut 的设计;检测实验可略。

  3. 3

    属于网络剪枝一类方法,说明剪枝能在不损失精度的前提下压缩网络。

    读抓剪枝与恢复相结合的思路;对比目标论文的迭代剪枝。

  4. 4
    Understanding deep learning requires rethinking generalization理解深度学习需要重新思考泛化 · 2016

    说明过参数化网络的泛化问题,对应彩票假设中大网络里藏着小网络的讨论背景。

    读抓随机标签实验及其对「容量与泛化」的含义。

  5. 5
    The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks彩票假设:寻找稀疏且可训练的神经网络 · 2018 · 就是这篇

    稠密网络中存在能单独训练到同等精度的稀疏子网络。

    读抓迭代幅度剪枝加回到原始初始化的流程,以及重新随机初始化后效果变差的对照。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
强烈推荐sanxiyn · 2019-05

“This is in fact the most interesting hypothesis on why neural networks work I have ever read.”

称这是读过的关于神经网络为何有效最有意思的假说
Neural nets typically contain smaller “subnetworks” that can often learn faster原评论 ↗
强烈推荐stared · 2022-02

“In the LTH paper (IMHO the most fundamental deep learning publication in the last few years), the number of tickets goes as layer_size^n_layers.”

称它是近几年最根本的深度学习论文,并用它推算 ticket 的数量
Computer scientists prove why bigger neural networks do better原评论 ↗
推荐stared · 2020-03

“I recommend for an overview:”

推荐作为概览,解释过参数化深度网络为何训练得好
Proving the Lottery Ticket Hypothesis: Pruning is All You Need原评论 ↗
推荐stfwn · 2020-03

“The lottery ticket hypothesis paper is good place to start”

推荐作为入门起点,剪枝可缩小网络 90% 以上而性能不降
Ask HN: What projects are you working on now?原评论 ↗
推荐nl · 2020-12

“it's either going to be a curiosity that people read, have their mind exploded but can't do anything with or else it's a good chance to be the most influential deep learning paper of the decade.”

认为它要么只是令人震撼的奇闻,要么可能成为这十年最有影响力的深度学习论文
Ask HN: What's the best paper you've read in 2020?原评论 ↗
推荐kylevedder · 2021-09

“You should look at Frankle's work on The Lottery Ticket Hypothesis [1];”

建议看 Frankle 的这项工作,多数权重可去掉而质量相近
Deep Learning's Diminishing Returns原评论 ↗
看全部 20 条 →
引用dplarson · 2019-05

“The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks”

贴出论文标题,指彩票假设:稠密网络中藏有稀疏可训练的子网络
Smarter Training of Neural Networks原评论 ↗
引用fromthestart · 2019-05

“based on the so called lottery ticket hypothesis [1]”

提到基于彩票假设,子网络用约 20% 参数即可达到完整精度
Notes on Weight Initialization for Deep Neural Networks原评论 ↗
引用nestorD · 2019-05

“If I understand the lottery ticket hypothesis [0] correctly, this would lead to a better exploration of the space and thus better results.”

按自己对彩票假设的理解,认为这样会更好地探索空间、效果更好
Notes on Weight Initialization for Deep Neural Networks原评论 ↗
引用jsharf · 2019-06

“Does this remind anyone else of the lottery ticket hypothesis deep learning paper that was published recently?”

联想到彩票假设:网络远大于所需,迭代剪枝能找到子网络
No Brainer (2015)原评论 ↗
引用choppaface · 2019-08

“which found that deep nets (for vision) contain discriminative sub-networks at initialization time”

引用它发现视觉深度网络在初始化时就含有有判别力的子网络
Exploring Weight Agnostic Neural Networks原评论 ↗
引用orange3xchicken · 2020-05

“the original work this paper follows up (by the same group: The Lottery Ticket Hypothesis)”

指出本文是同一团队这篇原始 lottery ticket 论文的后续
Researchers unveil a pruning algorithm to shrink deep learning models原评论 ↗
引用Reelin · 2020-09

“However, a counterpoint to your specific example would be the lottery ticket hypothesis and related weight agnostic neural networks.”

拿彩票假设和 weight agnostic 网络,作为对方例子的反例
Learning@home hivemind – train large neural networks across the internet原评论 ↗
引用ricklamers · 2020-09

“Sample efficiency isn't increased according to The Lottery Ticket Hypothesis.”

引用它说明 winning ticket 保持精度,但样本效率并未提高
Learning@home hivemind – train large neural networks across the internet原评论 ↗
引用pretty_dumm_guy · 2020-12

“For example, Lottery ticket hypothesis seems to reduce spacial complexity”

举彩票假设为例,说明它似乎能降低空间复杂度
Researchers propose faster, more efficient alternatives to backpropagation原评论 ↗
引用gojomo · 2021-07

“Something about this reminds me of the”

被它勾起联想:剪枝后的子网络用原始初始化也能训练得好
Mammals dream about the world they are about to experience before they are born原评论 ↗
引用talolard · 2021-09

“Second, there is a very popular paper called "The lottery ticket hypothesis" [1] that in any network you can find subnetworks that work just as well.”

提到这篇很流行的论文:任何网络里都能找到同样好用的子网络
An overview of the theory of overparameterized machine learning原评论 ↗
引用kn8a · 2022-01

“Sorry I meant the 'lottery ticket' hypothesis.”

更正说法,指的是彩票假设:小子网络可匹配大网络的性能
The AI Research SuperCluster原评论 ↗
引用kristianp · 2023-01

“The study referenced is (2018):”

给出所指的 2018 年那项研究
I won an ML lottery ticket, then squandered it原评论 ↗
引用polygamous_bat · 2023-01

“What you are describing is known as the "lottery ticket hypothesis" [0] in the ML world”

指出对方所描述的现象在 ML 界叫彩票假设,已有研究
The Transformer Family原评论 ↗