xCCL: A Survey of Industry-Led Collective Communication Libraries for Deep Learning-Reference-Cited by-同舟云学术

xCCL: A Survey of Industry-Led Collective Communication Libraries for Deep Learning

Published:2023-02 Issue:1 Volume:38 Page:166-195
ISSN:1000-9000
Container-title:Journal of Computer Science and Technology
language:en
Short-container-title:J. Comput. Sci. Technol.

Author:

Weingram Adam,Li Yuke,Qi Hao,Ng Darren,Dai Liuyao,Lu Xiaoyi

Publisher

Springer Science and Business Media LLC

Subject

Computational Theory and Mathematics,Computer Science Applications,Hardware and Architecture,Theoretical Computer Science,Software

Link

https://link.springer.com/content/pdf/10.1007/s11390-023-2894-6.pdf

Reference93 articles.

1. Hwang K, Xu Z W. Scalable Parallel Computing: Technology, Architecture, Programming. McGraw-Hill, 1998.

2. Brown T B, Mann B, Ryder N et al. Language models are few-shot learners. In Proc. the 34th Int. Conf. Neural Information Processing Systems, Dec. 2020, pp.1877–1901.

3. Naumov M, Mudigere D, Shi H J M et al. Deep learning recommendation model for personalization and recommendation systems. arXiv: 1906.00091, 2019. https://arxiv.org/abs/1906.00091, Jan. 2023.

4. Bayatpour M, Chakraborty S, Subramoni H, Lu X Y, Panda D K. Scalable reduction collectives with data partitioning-based multi-leader design. In Proc. the 2017 Int. Conf. High Performance Computing, Networking, Storage and Analysis (SC), Nov. 2017. https://doi.org/10.1145/3126908.3126954.

5. Chu C H, Lu X Y, Awan A A, Subramoni H, Hashmi J, Elton B, Panda D K. Efficient and scalable multi-source streaming broadcast on GPU clusters for deep learning. In Proc. the 46th Int. Conf. Parallel Processing (ICPP), Aug. 2017, pp.161–170. https://doi.org/10.1109/ICPP.2017.25.

Cited by 7 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. TCCL: Co-optimizing Collective Communication and Traffic Routing for GPU-centric Clusters;Proceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing;2024-08-04

2. Network Load Balancing with Parallel Flowlets for AI Training Clusters;Proceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing;2024-08-04

3. Collective Communication Performance Evaluation for Distributed Deep Learning Training;Applied Sciences;2024-06-12

4. Accuracy-Constrained Efficiency Optimization and GPU Profiling of CNN Inference for Detecting Drainage Crossing Locations;Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis;2023-11-12

5. Pareto Optimization of CNN Models via Hardware-Aware Neural Architecture Search for Drainage Crossing Classification on Resource-Limited Devices;Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis;2023-11-12