按年被提到
20152017 · 2 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1506.02626 · cs.NE

Learning both Weights and Connections for Efficient Neural Networks

Song Han, Jeff Pool, John Tran, William J. Dally · 2015
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 3 人首次 2016-12-13 · 最近 2017-12-13
3不同的人
2活跃月份
3讨论串
1,478被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems. Also, conventional networks fix the architecture before training starts; as a result, training cannot improve the architecture. To address these limitations, we describe a method to reduce the storage and computation required by neural networks by an order of magnitude without affecting their accuracy by learning only the important connections. Our method prunes redundant connections using a three-step method. First, we train the network to learn which connections are important. Next, we prune the unimportant connections. Finally, we retrain the network to fine tune the weights of the remaining connections. On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9x, from 61 million to 6.7 million, without incurring accuracy loss. Similar experiments with VGG-16 found that the number of parameters can be reduced by 13x, from 138 million to 10.3 million, again with no loss of accuracy.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论cing · 2016-12

“I believe several papers have examined efficiently pruning neural networks, but neural death would be better branding;) (https://arxiv.org/pdf/1506.02626v3.pdf https://arxiv.org/pdf/1510.00149v5.pdf)”

Neurogenesis Deep Learning原评论 ↗
评论sanxiyn · 2017-12

“This is the idea behind "Learning both Weights and Connections for Efficient Neural Networks (2015)", and yes, it works: https://arxiv.org/abs/1506.02626 How to figure out which ones are useful and which ones aren't? Why, you can try the simplest thing that could possibly work. Quoting the paper: "All connections with weights below a threshold are remove…”

Block-Sparse GPU Kernels原评论 ↗
评论trhway · 2017-12

“similar work from couple of years ago https://arxiv.org/pdf/1506.02626.pdf "our method prunes redundant connections using a three-step method. First, we train the network to learn which connections are important. Next, we prune the unim- portant connections. Finally, we retrain the network to fine tune the weights of the remaining connections. On the Ima…”

Pruning AI networks without impacting performance原评论 ↗