按年被提到
20162016 · 6 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1603.07341 · cs.LG

Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices

Tayfun Gokmen, Yurii Vlasov · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 6 人首次 2016-03-25 · 最近 2016-10-02
6不同的人
2活跃月份
6讨论串
486被引(OpenAlex)
arXiv 摘要(候选层不写导读)

In recent years, deep neural networks (DNN) have demonstrated significant business impact in large scale analysis and classification tasks such as speech recognition, visual object detection, pattern extraction, etc. Training of large DNNs, however, is universally considered as time consuming and computationally intensive task that demands datacenter-scale computational resources recruited for many days. Here we propose a concept of resistive processing unit (RPU) devices that can potentially accelerate DNN training by orders of magnitude while using much less power. The proposed RPU device can store and update the weight values locally thus minimizing data movement during training and allowing to fully exploit the locality and the parallelism of the training algorithm. We identify the RPU device and system specifications for implementation of an accelerator chip for DNN training in a realistic CMOS-compatible technology. For large DNNs with about 1 billion weights this massively parallel RPU architecture can achieve acceleration factors of 30,000X compared to state-of-the-art microprocessors while providing power efficiency of 84,000 GigaOps/s/W. Problems that currently require days of training on a datacenter-size cluster with thousands of machines can be addressed within hours on a single RPU accelerator. A system consisted of a cluster of RPU accelerators will be able to tackle Big Data problems with trillions of parameters that is impossible to address today like, for example, natural speech recognition and translation between all world languages, real-time analytics on large streams of business and scientific data, integration and analysis of multimodal sensory data flows from massive number of IoT (Internet of Things) sensors.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论cbennett · 2016-10

“Glad to hear you find it exciting. I do too.. its a really hot field at the moment and I mean that in the good, not bad way;) Lots of groups working in parallel on somewhat orthogonal design and architecture issues, with a variety of different considered devices, but a common basis set is emerging;) So, here's the paper I mentioned above. I think this …”

Researchers make a “diffusive” memristor that emulates how a real synapse work原评论 ↗
被提交为 story · 5 次