Analyzing the impact of the MPI allreduce in distributed training of convolutional neural networks-Reference-Cited by-同舟云学术

Analyzing the impact of the MPI allreduce in distributed training of convolutional neural networks

Published:2022-01-10 Issue:5 Volume:105 Page:1101-1119
ISSN:0010-485X
Container-title:Computing
language:en
Short-container-title:Computing

Author:

Castelló Adrián^ORCID,Catalán Mar,Dolz Manuel F.,Quintana-Ortí Enrique S.,Duato José

Funder

Ministerio de Ciencia, Innovación y Universidades

Generalitat Valenciana

Publisher

Springer Science and Business Media LLC

Subject

Computational Mathematics,Computational Theory and Mathematics,Computer Science Applications,Numerical Analysis,Theoretical Computer Science,Software

Link

https://link.springer.com/content/pdf/10.1007/s00607-021-01029-2.pdf

Reference19 articles.

1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X(2016) Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp 265–283, Savannah, GA, Nov. USENIX Association

2. Ben-Nun T, Hoefler T (2019) Demystifying parallel and distributed deep learning: An in-depth concurrency analysis. ACM Computing Surveys, 52(4):65:1–65:43

3. Castelló A, Dolz MF, Quintana-Ortí ES, Duato J (2019) Theoretical scalability analysis of distributed deep convolutional neural networks. In 2nd High Performance Machine Learning Workshop – HPML’2019, pp 534–541

4. Castelló A, Catalán M, Dolz MF, Mestre JI, Quintana-Ortí ES, Duato J (2021) Evaluation of mpi allreduce for distributed training of convolutional neural networks. In 2021 29th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pages 109–116

5. Chan E, Heimlich M, Purkayastha A, van de Geijn R (2007) Collective communication: Theory, practice, and experience. Concurr Comput: Practice Experience 19(13):1749–1783

Cited by 2 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. SUARA: A scalable universal allreduce communication algorithm for acceleration of parallel deep learning applications;Journal of Parallel and Distributed Computing;2024-01

2. Hybrid Parameter Update: Alleviating Imbalance Impacts for Distributed Deep Learning;2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys);2022-12