按年被提到
20172017 · 1 次2026
机器学习与深度学习›训练加速与低精度计算·arXiv:1711.05979 · cs.DC

Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs

Shaohuai Shi, Qiang Wang, Xiaowen Chu · 2017
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2017-11-21 · 最近 2018-12-08
2不同的人
2活跃月份
2讨论串
121被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Deep learning frameworks have been widely deployed on GPU servers for deep learning applications in both academia and industry. In training deep neural networks (DNNs), there are many standard processes or algorithms, such as convolution and stochastic gradient descent (SGD), but the running performance of different frameworks might be different even running the same deep model on the same GPU hardware. In this study, we evaluate the running performance of four state-of-the-art distributed deep learning frameworks (i.e., Caffe-MPI, CNTK, MXNet, and TensorFlow) over single-GPU, multi-GPU, and multi-node environments. We first build performance models of standard processes in training DNNs with SGD, and then we benchmark the running performance of these frameworks with three popular convolutional neural networks (i.e., AlexNet, GoogleNet and ResNet-50), after that, we analyze what factors that result in the performance gap among these four frameworks. Through both analytical and experimental analysis, we identify bottlenecks and overheads which could be further optimized. The main contribution is that the proposed performance models and the analysis provide further optimization directions in both algorithmic design and system configuration.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论stochastic_monk · 2018-12

“I’m also excited that Caffe got merged in because it has nearly perfect GPU-scaling via MPI, whereas tf suffers (or did, when [0] came out) with more GPUs. [0] https://arxiv.org/pdf/1711.05979”

PyTorch 1.0 is out原评论 ↗
被提交为 story · 1 次