按年被提到
20192021 · 1 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1912.05234 · cs.PL

Array Languages Make Neural Networks Fast

Artjoms Šinkarovs, Hans-Nikolai Vießmann, Sven-Bodo Scholz · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2021-04-09 · 最近 2023-03-08
2不同的人
2活跃月份
1讨论串
5被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Modern machine learning frameworks are complex: they are typically organised in multiple layers each of which is written in a different language and they depend on a number of external libraries, but at their core they mainly consist of tensor operations. As array-oriented languages provide perfect abstractions to implement tensor operations, we consider a minimalistic machine learning framework that is shallowly embedded in an array-oriented language and we study its productivity and performance. We do this by implementing a state of the art Convolutional Neural Network (CNN) and compare it against implementations in TensorFlow and PyTorch --- two state of the art industrial-strength frameworks. It turns out that our implementation is 2 and 3 times faster, even after fine-tuning the TensorFlow and PyTorch to our hardware --- a 64-core GPU-accelerated machine. The size of all three CNN specifications is the same, about 150 lines of code. Our mini framework is 150 lines of highly reusable hardware-agnostic code that does not depend on external libraries. The compiler for a host array language automatically generates parallel code for a chosen architecture. The key to such a balance between performance and portability lies in the design of the array language; in particular, the ability to express rank-polymorphic operations concisely, yet being able to do optimisations across them. This design builds on very few assumptions, and it is readily transferable to other contexts offering a clean approach to high-performance machine learning.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论polyrand · 2021-04

“This reminds me of this paper that also caught my attention. They used an array language to make convolutions run very fast on CPUs. Array Languages Make Neural Networks Fast: https://arxiv.org/abs/1912.05234”

CPU-based algorithm trains deep neural nets up to 15 times faster than top GPU原评论 ↗
评论rramadass · 2023-03

“Thanks for the pointers, will definitely look into it. >The array model is great, but when running on von Neumann architectures you lose execution potential I expected that the array model would be best for SIMD and GPU parallel processing and as such a good fit for ML. See Also: Array Languages Make Neural Networks Fast - https://arxiv.org/pdf/1912.05234”