按年被提到
20242024 · 7 次2026
大模型与语言模型›混合专家与高效训练·arXiv:2406.02528 · cs.CL

Scalable MatMul-free Language Modeling

Rui-Jie Zhu, Yu Zhang, Steven Abreu, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Sumit Bam Shrestha, Peng Zhou, Jason K. Eshraghian · 2024
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 7 人首次 2024-06-07 · 最近 2024-10-03
7不同的人
2活跃月份
6讨论串
24被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large Language Models (LLMs) have fundamentally altered how we approach scaling in machine learning. However, these models pose substantial computational and memory challenges, primarily due to the reliance on matrix multiplication (MatMul) within their attention and feed-forward (FFN) layers. We demonstrate that MatMul operations can be eliminated from LLMs while maintaining strong performance, even at billion-parameter scales. Our MatMul-free models, tested on models up to 2.7B parameters, are comparable to state-of-the-art pre-trained Transformers, and the performance gap narrows as model size increases. Our approach yields significant memory savings: a GPU-efficient implementation reduces memory consumption by up to 61% during training and over 10x during inference. When adapted for a multi-chip neuromorphic system, the model leverages asynchronous processing to achieve 4x higher throughput with 10x less energy than edge GPUs.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论cs702 · 2024-06

“Paper: https://arxiv.org/abs/2406.02528 -- always better than a press release. Code: https://github.com/ridgerchu/matmulfreellm --- Like others before them, the authors train LLMs using parameters consisting of ternary digits, or trits, with values in {-1, 0, 1}. What's new is that the authors then build a custom hardware solution on an FPGA and run b…”

Researchers run high-performing LLM on the energy needed to power a lightbulb原评论 ↗
评论Aurornis · 2024-06

“The press release is devoid of useful information, unsurprisingly. You can run an LLM under almost any energy envelope if you’re willing to wait long enough for the result. Total energy consumed and the time difference are the more important metrics. The actual paper is here: https://arxiv.org/abs/2406.02528 The key part from the summary: > To properly…”

Researchers run high-performing LLM on the energy needed to power a lightbulb原评论 ↗
评论tomohelix · 2024-06

“The relevant paper: https://arxiv.org/abs/2406.02528 In summary, they forced the model to process data in ternary system and then build a custom FPGA chip to process the data more efficiently. Tested to be "comparable" to small models (3B), theoretically scale to 70B, unknown for SOTAs (>100B params). We have always known custom chips are more efficient…”

Researchers upend AI status quo by eliminating matrix multiplication in LLMs原评论 ↗
评论akrymski · 2024-10

“Fantastic result, on par with another similar effort: https://arxiv.org/pdf/2406.02528 It seems to me that we've stumbled upon this method of GPU-heavy matrix-multiplications in deep neural nets, and have only scratched the surface of alternative methods that are actually optimized for current CPU architectures such as Tsetlin Machines, Hyperdimensional …”

Addition is All You Need for Energy-efficient Language Models原评论 ↗
被提交为 story · 3 次