“Lots of people doing interesting work in that area, check out these guys - [LINK]”
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
把卷积网络权重甚至输入二值化,省内存且可在 CPU 实时运行
论文提出对标准卷积神经网络的两种高效近似:Binary-Weight-Networks 把滤波器近似为二值,节省 32 倍内存;XNOR-Networks 则让滤波器和卷积层输入都是二值,主要用二值运算近似卷积,卷积运算快 58 倍、内存节省 32 倍,使 CPU 上实时运行先进网络成为可能。在 ImageNet 上,Binary-Weight 版 AlexNet 的 top-1 准确率仅比全精度低 2.9%,并比 BinaryConnect 和 BinaryNets 高出超过 16%。
HN 上的人把它当作 1 比特量化网络的早期代表,在讨论低精度推理、硬件加速器和后来的二值网络工作时反复引用,用来说明极低精度是可行的。
学习路径
读这篇之前,按顺序读书架上的 7 篇 · 参考文献来自 OpenAlex,顺序和要点由 Claude Code 排出这条路径先看为什么要压缩网络(浅网络、剪枝、小模型),再看二值权重的前作 BinaryConnect,最后读 XNOR-Net 如何把输入也二值化。
- 1
- 2
剪枝是与二值化并列的压缩路线,可用来比较省内存的方法。
读按权重大小剪枝再重训练的流程;了解即可。
- 3
同样追求小模型,是 XNOR-Net 论文里比较压缩效果的对象。
读Fire 模块的设计与参数量与精度的关系。
- 4
- 5
XNOR-Net 的 Binary-Weight-Network 直接建立在它的二值权重训练之上。
读前向用二值权重、反向用实数权重更新的训练方法。
- 6
XNOR-Net 在二值卷积前做批归一化,需要理解其作用。
读批归一化的计算方式与作用,对照 XNOR-Net 里的层顺序。
- 7
- 8XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
目标论文:Binary-Weight-Networks 与 XNOR-Networks。
读二值权重的缩放因子最优解的推导,以及用 XNOR 加位计数近似卷积;再看 ImageNet 实验。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“it'd be interesting to see how this technique stacks up against one of those fancy new binary networks (e.g. XNOR-Net)”
“Arguably, one could get another order of magnitude speedup by implementing XNOR-net hardware accelerator, but it hasn't been done yet.”
“The main difference is that XNOR-Net uses 1 bit quantized weights and XNOR+popcount to approximate the dot product of convolution”
“Even smaller precision is practical (down to 1 bit with XNORnet[2]”
“There was no evidence for his claim then, and now this is clearly false [2, 3].”
看全部 7 条 →
“Exactly as xnornet was doing way back in 2016 - shadow 32bit weights, quantized to 1 bit during the forward pass.”
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks ↗jarmitage · 2016-03-19 · 74 分