Enhanced Scheduling of AI Applications in Multi-Tenant Cloud Using Genetic Optimizations-Reference-Cited by-同舟云学术

Enhanced Scheduling of AI Applications in Multi-Tenant Cloud Using Genetic Optimizations

Published:2024-05-29 Issue:11 Volume:14 Page:4697
ISSN:2076-3417
Container-title:Applied Sciences
language:en
Short-container-title:Applied Sciences

Author:

Kwon Seokmin¹,Bahn Hyokyung¹^ORCID

Affiliation:

1. Department of Computer Engineering, Ewha University, Seoul 03760, Republic of Korea

Abstract

The artificial intelligence (AI) industry is increasingly integrating with diverse sectors such as smart logistics, FinTech, entertainment, and cloud computing. This expansion has led to the coexistence of heterogeneous applications within multi-tenant systems, presenting significant scheduling challenges. This paper addresses these challenges by exploring the scheduling of various machine learning workloads in large-scale, multi-tenant cloud systems that utilize heterogeneous GPUs. Traditional scheduling strategies often struggle to achieve satisfactory results due to low GPU utilization in these complex environments. To address this issue, we propose a novel scheduling approach that employs a genetic optimization technique, implemented within a process-oriented discrete-event simulation framework, to effectively orchestrate various machine learning tasks. We evaluate our approach using workload traces from Alibaba’s MLaaS cluster with over 6000 heterogeneous GPUs. The results show that our scheduling improves GPU utilization by 12.8% compared to Round-Robin scheduling, demonstrating the effectiveness of the solution in optimizing cloud-based GPU scheduling.

Funder

Institute of Information and Communications Technology Planning and Evaluation

Artificial Intelligence Convergence Innovation Human Resources Development

Publisher

MDPI AG

Link

https://www.mdpi.com/2076-3417/14/11/4697/pdf

Reference31 articles.

1. Weng, Q., Xiao, W., Yu, Y., Wang, W., Wang, C., He, J., Li, Y., Zhang, L., Lin, W., and Ding, Y. (2022, January 4–6). MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters. Proceedings of the 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22), USENIX Association, Renton, WA, USA.

2. Jeon, M., Venkataraman, S., Phanishayee, A., Qian, J., Xiao, W., and Yang, F. (2019, January 10–12). Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads. Proceedings of the 2019 USENIX Annual Technical Conference (USENIX ATC 19), Renton, WA, USA.

3. Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., Fawzy, M., Jia, B., Jia, Y., and Kalro, A. (2018, January 24–28). Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective. Proceedings of the 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA), IEEE, Vienna, Austria.

4. Mohan, J., Phanishayee, A., Kulkarni, J., and Chidambaram, V. (2022, January 11–13). Looking Beyond GPUs for DNN Scheduling on Multi-Tenant Clusters. Proceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), USENIX Association, Carlsbad, CA, USA.

5. Zhao, H., Han, Z., Yang, Z., Zhang, Q., Yang, F., Zhou, L., Yang, M., Lau, F.C., Wang, Y., and Xiong, Y. (2020, January 4–6). HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees. Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), Online.