A generic communication scheduler for distributed DNN training acceleration-Reference-Cited by-同舟云学术

A generic communication scheduler for distributed DNN training acceleration

Published:2019-10-27 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 27th ACM Symposium on Operating Systems Principles
language:
Short-container-title:

Author:

Peng Yanghua¹,Zhu Yibo²,Chen Yangrui³,Bao Yixin³,Yi Bairen²,Lan Chang²,Wu Chuan²,Guo Chuanxiong³

Affiliation:

1. The University of Hong Kong and ByteDance Inc.

2. ByteDance Inc.

3. The University of Hong Kong

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3341301.3359642

Reference39 articles.

1. 2019. ByteScheduler Appendix. https://www.dropbox.com/s/smoq6xd6pr7av81/bytescheduler_appendix.pdf?dl=0. 2019. ByteScheduler Appendix. https://www.dropbox.com/s/smoq6xd6pr7av81/bytescheduler_appendix.pdf?dl=0.

2. 2019. ByteScheduler Source Code. https://github.com/bytedance/byteps. 2019. ByteScheduler Source Code. https://github.com/bytedance/byteps.

3. 2019. MLPerf Training v0.6 Results. https://mlperf.org/training-results-0-6/. 2019. MLPerf Training v0.6 Results. https://mlperf.org/training-results-0-6/.

4. 2019. NVIDIA Collective Communications Library (NCCL). https://developer.nvidia.com/nccl. 2019. NVIDIA Collective Communications Library (NCCL). https://developer.nvidia.com/nccl.

5. 2019. TensorFlow Grapper. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/grappler. 2019. TensorFlow Grapper. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/grappler.

Cited by 198 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. WBSP: Addressing stragglers in distributed machine learning with worker-busy synchronous parallel;Parallel Computing;2024-09

2. Efficient Inter-Datacenter AllReduce With Multiple Trees;IEEE Transactions on Network Science and Engineering;2024-09

3. AutoPipe: Automatic Configuration of Pipeline Parallelism in Shared GPU Cluster;Proceedings of the 53rd International Conference on Parallel Processing;2024-08-12

4. Sparse Gradient Communication with AlltoAll for Accelerating Distributed Deep Learning;Proceedings of the 53rd International Conference on Parallel Processing;2024-08-12

5. SqueezeNIC: Low-Latency In-NIC Compression for Distributed Deep Learning;Proceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing;2024-08-04