Mitigating Stragglers in the Decentralized Training on Heterogeneous Clusters-Reference-Cited by-同舟云学术

Mitigating Stragglers in the Decentralized Training on Heterogeneous Clusters

Published:2020-12-07 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 21st International Middleware Conference
language:
Short-container-title:

Author:

Yang Donglin¹,Rang Wei¹,Cheng Dazhao¹

Affiliation:

1. University of North Carolina at Charlotte, Charlotte, NC, USA

Funder

National Science Foundation

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3423211.3425693

Reference66 articles.

1. Jimmy Ba and Brendan Frey. 2013. Adaptive dropout for training deep neural networks. In Advances in neural information processing systems. 3084--3092. Jimmy Ba and Brendan Frey. 2013. Adaptive dropout for training deep neural networks. In Advances in neural information processing systems. 3084--3092.

2. Round-Robin Synchronization: Mitigating Communication Bottlenecks in Parameter Servers

Cited by 9 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. MPMoE: Memory Efficient MoE for Pre-Trained Models With Adaptive Pipeline Parallelism;IEEE Transactions on Parallel and Distributed Systems;2024-06

2. Heter-Train: A Distributed Training Framework Based on Semi-Asynchronous Parallel Mechanism for Heterogeneous Intelligent Transportation Systems;IEEE Transactions on Intelligent Transportation Systems;2024-01

3. Adaptive Worker Group Management for Mitigating Straggler Problem in Distributed Deep Learning Systems;2024

4. Chronica: A Data-Imbalance-Aware Scheduler for Distributed Deep Learning;2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid);2023-05

5. FSP: Towards Flexible Synchronous Parallel Frameworks for Distributed Machine Learning;IEEE Transactions on Parallel and Distributed Systems;2023-02-01