按年被提到
20162016 · 1 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1603.01025 · cs.NE

Convolutional Neural Networks using Logarithmic Data Representation

Daisuke Miyashita, Edward H. Lee, Boris Murmann · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2016-10-11 · 最近 2025-06-20
2不同的人
2活跃月份
2讨论串
344被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Recent advances in convolutional neural networks have considered model complexity and hardware efficiency to enable deployment onto embedded systems and mobile devices. For example, it is now well-known that the arithmetic operations of deep networks can be encoded down to 8-bit fixed-point without significant deterioration in performance. However, further reduction in precision down to as low as 3-bit fixed-point results in significant losses in performance. In this paper we propose a new data representation that enables state-of-the-art networks to be encoded to 3 bits with negligible loss in classification performance. To perform this, we take advantage of the fact that the weights and activations in a trained network naturally have non-uniform distributions. Using non-uniform, base-2 logarithmic representation to encode weights, communicate activations, and perform dot-products enables networks to 1) achieve higher classification accuracies than fixed-point at the same resolution and 2) eliminate bulky digital multipliers. Finally, we propose an end-to-end training procedure that uses log representation at 5-bits, which achieves higher final test accuracy than linear at 5-bits.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论p1esk · 2016-10

“FP16 performance is only relevant until people figure out how to train NNs using INT8. See, for example, [1] for recent advances in that direction. After that, it's going to be mostly about memory size and bandwidth. [1] https://arxiv.org/abs/1603.01025”

OpenAI is Using Reddit to Teach An Artificial Intelligence How to Speak原评论 ↗
评论fc417fc802 · 2025-06

“It's been done. [0][1] But the fact that BitNet [2] is possible calls into question the need for it. NNs are surprisingly robust to a wide array of distortions. [3] [0] https://arxiv.org/abs/1603.01025 [1] https://par.nsf.gov/servlets/purl/10163070 [2] https://arxiv.org/abs/2310.11453 [3] https://arxiv.org/abs/1606.01981”

Posit floating point numbers: thin triangles and other tricks (2019)原评论 ↗