Efficient sparse collective communication and its application to accelerate distributed deep learning-Reference-Cited by-同舟云学术

Efficient sparse collective communication and its application to accelerate distributed deep learning

Published:2021-08-09 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 2021 ACM SIGCOMM 2021 Conference
language:
Short-container-title:

Author:

Fei Jiawei¹,Ho Chen-Yu²,Sahu Atal N.²,Canini Marco²,Sapio Amedeo³

Affiliation:

1. NUDT and KAUST

2. KAUST

3. Intel

Funder

King Abdullah University of Science and Technology

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3452296.3472904

Reference72 articles.

1. Alham Fikri Aji and Kenneth Heafield. 2017. Sparse Communication for Distributed Gradient Descent. In EMNLP-IJCNLP. Alham Fikri Aji and Kenneth Heafield. 2017. Sparse Communication for Distributed Gradient Descent. In EMNLP-IJCNLP .

2. Dan Alistarh Torsten Hoefler Mikael Johansson Nikola Konstantinov Sarit Khirirat and Cédric Renggli. 2018. The Convergence of Sparsified Gradient Methods. In NeurIPS. Dan Alistarh Torsten Hoefler Mikael Johansson Nikola Konstantinov Sarit Khirirat and Cédric Renggli. 2018. The Convergence of Sparsified Gradient Methods. In NeurIPS .

3. P4

Cited by 48 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. When In-Network Computing Meets Distributed Machine Learning;IEEE Network;2024-09

2. OmNICCL: Zero-cost Sparse AllReduce with Direct Cache Access and SmartNICs;Proceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing;2024-08-04

3. Accelerating Model Training in Multi-cluster Environments with Consumer-grade GPUs;Proceedings of the ACM SIGCOMM 2024 Conference;2024-08-04

4. Accelerating Distributed Training With Collaborative In-Network Aggregation;IEEE/ACM Transactions on Networking;2024-08

5. AutoDDL: Automatic Distributed Deep Learning With Near-Optimal Bandwidth Cost;IEEE Transactions on Parallel and Distributed Systems;2024-08