ETS: Deep Learning Training Iteration Time Prediction based on Execution Trace Sliding Window-Reference-Cited by-同舟云学术

ETS: Deep Learning Training Iteration Time Prediction based on Execution Trace Sliding Window

Published:2024-06-03 Issue: Volume:33 Page:56-68
ISSN:
Container-title:Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing
language:
Short-container-title:

Author:

Yang Zichao¹^ORCID,Guo Hao¹^ORCID,Wu Heng²³⁴⁵^ORCID,Wu Yuewen²^ORCID,Zhong Hua²^ORCID,Zhang Wenbo²³⁴^ORCID,Zhou Chuan⁶^ORCID,Liu Yan⁷^ORCID

Affiliation:

1. University of Chinese Academy of Sciences, Beijing, China

2. Institute of Software, Chinese Academy of Sciences, Beijing, China

3. University of Chinese Academy of Sciences, Nanjing, China

4. Nanjing Institute of Software Technology, Nanjing, China

5. Key Laboratory of System Software (Chinese Academy of Sciences) and State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences., Beijing, China

6. Minzu University of China, Beijing, China

7. Inspur Software Co., Ltd., Jinan, China

Funder

National Key Research and Development Program of China

National Natural Science Foundation of China

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3625549.3658658

Reference56 articles.

1. 2020. Paddle Quantum. https://github.com/PaddlePaddle/Quantum

2. 2022. BERT. https://github.com/google-research/bert

3. 2024. NVIDIA Nsight Systems. https://developer.nvidia.com/nsight-systems

4. Martín Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dandelion Mané Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Viégas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.

5. Lu Bai, Weixing Ji, Qinyuan Li, Xilai Yao, Wei Xin, and Wanyi Zhu. 2022. Dnnabacus: Toward accurate computational cost prediction for deep neural networks. arXiv preprint arXiv:2205.12095 (2022).