GEMINI: Fast Failure Recovery in Distributed Training with In-Memory Checkpoints-Reference-Cited by-同舟云学术

GEMINI: Fast Failure Recovery in Distributed Training with In-Memory Checkpoints

Published:2023-10-23 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 29th Symposium on Operating Systems Principles
language:
Short-container-title:

Author:

Wang Zhuang¹^ORCID,Jia Zhen²^ORCID,Zheng Shuai³^ORCID,Zhang Zhen³^ORCID,Fu Xinwei³^ORCID,Ng T. S. Eugene⁴^ORCID,Wang Yida³^ORCID

Affiliation:

1. Department of Computer Science, Rice University, Houston, Texas, United States of America

2. Amazon Web Services, Santa Clara, California, USA

3. Amazon Web Services, Santa Clara, California, United States of America

4. Rice University, Houston, Texas, United States of America

Funder

NSF (National Science Foundation)

Publisher

ACM

Reference89 articles.

1. NVIDIA NCCL. https://developer.nvidia.com/NCCL , 2021 . NVIDIA NCCL. https://developer.nvidia.com/NCCL, 2021.

2. AWS Trainium . https://aws.amazon.com/machine-learning/trainium/ , 2022 . AWS Trainium. https://aws.amazon.com/machine-learning/trainium/, 2022.

3. BLOOM Chronicles . https://github.com/bigscience-workshop/bigscience/blob/master/train/tr11-176B-ml/chronicles.md , 2022 . BLOOM Chronicles. https://github.com/bigscience-workshop/bigscience/blob/master/train/tr11-176B-ml/chronicles.md, 2022.

4. Auto Scaling in AWS. https://docs.aws.amazon.com/autoscaling/ , 2023 . Auto Scaling in AWS. https://docs.aws.amazon.com/autoscaling/, 2023.

5. Auto Scaling in Azure . https://learn.microsoft.com/en-us/azure/app-service/manage-scale-up , 2023 . Auto Scaling in Azure. https://learn.microsoft.com/en-us/azure/app-service/manage-scale-up, 2023.

Cited by 6 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. LubeRDMA: A Fail-safe Mechanism of RDMA;Proceedings of the 8th Asia-Pacific Workshop on Networking;2024-08-03

2. Portus: Efficient DNN Checkpointing to Persistent Memory with Zero-Copy;2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS);2024-07-23

3. Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures;Proceedings of the Nineteenth European Conference on Computer Systems;2024-04-22

4. Optimizing Shared File Access in Distributed Parallel File System;2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT);2024-03-29

5. PancakeFS: A Write Efficiently and Read Optimized Filesystem;2024 4th International Conference on Consumer Electronics and Computer Engineering (ICCECE);2024-01-12