按年被提到
20242024 · 8 次2026
大模型与语言模型›混合专家与高效训练·arXiv:2410.00907 · cs.CL

Addition is All You Need for Energy-efficient Language Models

Hongyin Luo, Wei Sun · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 8 人首次 2024-10-02 · 最近 2024-12-04
8不同的人
2活跃月份
8讨论串
9被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large neural networks spend most computation on floating point tensor multiplications. In this work, we find that a floating point multiplier can be approximated by one integer adder with high precision. We propose the linear-complexity multiplication L-Mul algorithm that approximates floating point number multiplication with integer addition operations. The new algorithm costs significantly less computation resource than 8-bit floating point multiplication but achieves higher precision. Compared to 8-bit floating point multiplications, the proposed method achieves higher precision but consumes significantly less bit-level computation. Since multiplying floating point numbers requires substantially higher energy compared to integer addition operations, applying the L-Mul operation in tensor processing hardware can potentially reduce 95% energy cost by element-wise floating point tensor multiplications and 80% energy cost of dot products. We calculated the theoretical error expectation of L-Mul, and evaluated the algorithm on a wide range of textual, visual, and symbolic tasks, including natural language understanding, structural reasoning, mathematics, and commonsense question answering. Our numerical analysis experiments agree with the theoretical error estimation, which indicates that L-Mul with 4-bit mantissa achieves comparable precision as float8_e4m3 multiplications, and L-Mul with 3-bit mantissa outperforms float8_e5m2. Evaluation results on popular benchmarks show that directly applying L-Mul to the attention mechanism is almost lossless. We further show that replacing all floating point multiplications with 3-bit mantissa L-Mul in a transformer model achieves equivalent precision as using float8_e4m3 as accumulation precision in both fine-tuning and inference.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论datalto · 2024-10

“Hi, We've been working on a solution to tackle one of AI’s growing challenges: energy consumption. Running large AI models, like those behind services such as ChatGPT, can use as much electricity as thousands of homes daily. This energy demand isn’t just costly—it has a massive environmental footprint. To address this, we developed an algorithm called L…”

评论djoldman · 2024-10

“https://arxiv.org/abs/2410.00907 ABSTRACT Large neural networks spend most computation on floating point tensor multiplications. In this work, we find that a floating point multiplier can be approximated by one integer adder with high precision. We propose the linear-complexity multiplication (L-Mul) algorithm that approximates floating point number mul…”

AI engineers claim new algorithm reduces AI power consumption by 95%原评论 ↗
评论dogcomplex · 2024-12

“I am favoriting this comment for reference later when I start poking around in the base level stuff. I find it pretty funny how simple this stuff can get. Have you messed with ternary computing inference yet? I imagine that shrinks the list even further - or at least reduces the compute requirements in favor of brute force addition. https://arxiv.org/html…”

Intel announces Arc B-series "Battlemage" discrete graphics with Linux support原评论 ↗
被提交为 story · 5 次