按年被提到
20232023 · 2 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:2303.07470 · cs.LG

X-Former: In-Memory Acceleration of Transformers

Shrihari Sridharan, Jacob R. Stevens, Kaushik Roy, Anand Raghunathan · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-03-17 · 最近 2023-05-13
2不同的人
2活跃月份
2讨论串
70被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Transformers have achieved great success in a wide variety of natural language processing (NLP) tasks due to the attention mechanism, which assigns an importance score for every word relative to other words in a sequence. However, these models are very large, often reaching hundreds of billions of parameters, and therefore require a large number of DRAM accesses. Hence, traditional deep neural network (DNN) accelerators such as GPUs and TPUs face limitations in processing Transformers efficiently. In-memory accelerators based on non-volatile memory promise to be an effective solution to this challenge, since they provide high storage density while performing massively parallel matrix vector multiplications within memory arrays. However, attention score computations, which are frequently used in Transformers (unlike CNNs and RNNs), require matrix vector multiplications (MVM) where both operands change dynamically for each input. As a result, conventional NVM-based accelerators incur high write latency and write energy when used for Transformers, and further suffer from the low endurance of most NVM technologies. To address these challenges, we present X-Former, a hybrid in-memory hardware accelerator that consists of both NVM and CMOS processing elements to execute transformer workloads efficiently. To improve the hardware utilization of X-Former, we also propose a sequence blocking dataflow, which overlaps the computations of the two processing elements and reduces execution time. Across several benchmarks, we show that X-Former achieves upto 85x and 7.5x improvements in latency and energy over a NVIDIA GeForce GTX 1060 GPU and upto 10.7x and 4.6x improvements in latency and energy over a state-of-the-art in-memory NVM accelerator.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论ilaksh · 2023-05

“I think the most interesting AI hardware stuff is about memristors or some type of compute-in-memory. https://arxiv.org/pdf/2303.07470.pdf https://ieeexplore.ieee.org/abstract/document/9669041 Maybe there will be something like transformers but more suited to crossbar arrays of memristors. If that actually makes sense.”

Google Launches AI Supercomputer Powered by Nvidia H100 GPUs原评论 ↗
被提交为 story · 1 次