Efficient Process Arrival Pattern Aware Collective Communication for Deep Learning-Reference-Cited by-同舟云学术

Efficient Process Arrival Pattern Aware Collective Communication for Deep Learning

Published:2022-09-14 Issue: Volume: Page:
ISSN:
Container-title:EuroMPI/USA'22: 29th European MPI Users' Group Meeting
language:
Short-container-title:

Author:

Alizadeh Pedram¹,Sojoodi Amirhossein¹,Hassan Temucin Yiltan¹,Afsahi Ahmad¹

Affiliation:

1. Queen's University, Canada

Funder

Compute Canada

Natural Sciences and Engineering Research Council of Canada

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3555819.3555857

Reference37 articles.

1. (June 2022). CIFAR-10 and CIFAR-100 datasets. https://www.cs.toronto.edu/ kriz/cifar.html (June 2022). CIFAR-10 and CIFAR-100 datasets. https://www.cs.toronto.edu/ kriz/cifar.html

2. (June 2022). Message Passing Interface (MPI 4.0). http://www.mpi-forum.org (June 2022). Message Passing Interface (MPI 4.0). http://www.mpi-forum.org

3. M. Abadi A. Agarwal P. Barham E. Brevdo Z. Chen C. Citro G. S. Corrado A. Davis J. Dean M. Devin S. Ghemawat I. Goodfellow A. Harp G. Irving M. Isard Y. Jia R. Jozefowicz L. Kaiser M. Kudlur J. Levenberg D. Mané R. Monga S. Moore D. Murray C. Olah M. Schuster J. Shlens B. Steiner I. Sutskever K. Talwar P. Tucker V. Vanhoucke V. Vasudevan F. Viégas O. Vinyals P. Warden M. Wattenberg M. Wicke Y. Yu and X. Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ M. Abadi A. Agarwal P. Barham E. Brevdo Z. Chen C. Citro G. S. Corrado A. Davis J. Dean M. Devin S. Ghemawat I. Goodfellow A. Harp G. Irving M. Isard Y. Jia R. Jozefowicz L. Kaiser M. Kudlur J. Levenberg D. Mané R. Monga S. Moore D. Murray C. Olah M. Schuster J. Shlens B. Steiner I. Sutskever K. Talwar P. Tucker V. Vanhoucke V. Vasudevan F. Viégas O. Vinyals P. Warden M. Wattenberg M. Wicke Y. Yu and X. Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/

4. Q. Ali , S. P. Midkiff , and V. S. Pai . 2009 . Efficient high performance collective communication for the Cell blade . In Proceedings of the 23rd international conference on Supercomputing (ICS). 193–203 . Q. Ali, S. P. Midkiff, and V. S. Pai. 2009. Efficient high performance collective communication for the Cell blade. In Proceedings of the 23rd international conference on Supercomputing (ICS). 193–203.

5. Demystifying Parallel and Distributed Deep Learning

Cited by 5 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Analysis and prediction of performance variability in large-scale computing systems;The Journal of Supercomputing;2024-03-28

2. A Dynamic Network-Native MPI Partitioned Aggregation Over InfiniBand Verbs;2023 IEEE International Conference on Cluster Computing (CLUSTER);2023-10-31

3. Synchronizing MPI Processes in Space and Time;Proceedings of the 30th European MPI Users' Group Meeting;2023-09-11

4. Analysis and Prediction of Performance Variability in Large-Scale Computing Systems;2023

5. A Visual Guide to MPI All-to-all;2022 IEEE 29th International Conference on High Performance Computing, Data and Analytics Workshop (HiPCW);2022-12