DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining-Reference-Cited by-同舟云学术

DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining

Published:2023-07 Issue: Volume: Page:
ISSN:
Container-title:2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS)
language:
Short-container-title:

Author:

Zhang Lin¹,Shi Shaohuai²,Chu Xiaowen³,Wang Wei¹,Li Bo¹,Liu Chengjian⁴

Affiliation:

1. The Hong Kong University of Science and Technology

2. Harbin Institute of Technology,Shenzhen

3. The Hong Kong University of Science and Technology (Guangzhou)

4. Shenzhen Technology University

Funder

National Natural Science Foundation of China (NSFC)

Publisher

IEEE

Link

http://xplorestaging.ieee.org/ielx7/10272385/10272393/10272398.pdf?arnumber=10272398

Reference54 articles.

1. Poseidon: An efficient communication architecture for distributed deep learning on GPU clusters;zhang;2017 USENIX Annual Technical Conference (USENIX ATC 17),2017

2. Performance Modeling and Evaluation of Distributed Deep Learning Frameworks on GPUs

3. PyTorch distributed

4. S-Caffe

5. Exploiting Simultaneous Communications to Accelerate Data Parallel Distributed Deep Learning

Cited by 3 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Sparse Gradient Communication with AlltoAll for Accelerating Distributed Deep Learning;Proceedings of the 53rd International Conference on Parallel Processing;2024-08-12

2. Centauri: Enabling Efficient Scheduling for Communication-Computation Overlap in Large Model Training via Communication Partitioning;Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3;2024-04-27

3. PARING: Joint Task Placement and Routing for Distributed Training With In-Network Aggregation;IEEE/ACM Transactions on Networking;2024