HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis-Reference-Cited by-同舟云学术

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

Published:2024-04-22 Issue: Volume:1 Page:524-541
ISSN:
Container-title:Proceedings of the Nineteenth European Conference on Computer Systems
language:
Short-container-title:

Author:

Zhang Shiwei¹^ORCID,Diao Lansong²^ORCID,Wu Chuan¹^ORCID,Cao Zongyan²^ORCID,Wang Siyu²^ORCID,Lin Wei²^ORCID

Affiliation:

1. The University of Hong Kong

2. Alibaba Group

Funder

Alibaba Group and Hong Kong RGC

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3627703.3629580

Reference59 articles.

1. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16).

2. Ravichandra Addanki, Shaileshh Bojja Venkatakrishnan, Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh. 2019. Placeto: Learning generalizable device placement algorithms for distributed machine learning. arXiv preprint arXiv:1906.08879 (2019).

3. Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2013. Syntax-guided synthesis. IEEE.

4. Rajeev Alur, Dana Fisman, Rishabh Singh, and Armando Solar-Lezama. 2017. Sygus-comp 2017: Results and analysis. arXiv preprint arXiv:1711.11438 (2017).

5. Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Dan Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, et al. 2022. Pathways: Asynchronous distributed dataflow for ML. arXiv preprint arXiv:2203.12533 (2022).