按年被提到
20152016 · 2 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1510.00149 · cs.CV

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Song Han, Huizi Mao, William J. Dally · 2015
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2016-01-11 · 最近 2016-12-13
2不同的人
2活跃月份
2讨论串
4,538被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论dharma1 · 2016-01

“Here's the paper - http://arxiv.org/pdf/1510.00149.pdf The compressed network achieves a decent speedup and energy saving on current hardware too (desktop/mobile) without significant loss of accuracy”

Emergent Chip Vastly Accelerates Deep Neural Networks原评论 ↗
评论cing · 2016-12

“I believe several papers have examined efficiently pruning neural networks, but neural death would be better branding;) (https://arxiv.org/pdf/1506.02626v3.pdf https://arxiv.org/pdf/1510.00149v5.pdf)”

Neurogenesis Deep Learning原评论 ↗