ElasticRoom: Multi-Tenant DNN Inference Engine via Co-design with Resource-constrained Compilation and Strong Priority Scheduling-Reference-Cited by-同舟云学术

ElasticRoom: Multi-Tenant DNN Inference Engine via Co-design with Resource-constrained Compilation and Strong Priority Scheduling

Published:2024-06-03 Issue: Volume:4 Page:1-14
ISSN:
Container-title:Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing
language:
Short-container-title:

Author:

Ma Lixian¹²^ORCID,Chen Haoruo¹²^ORCID,Shao En¹²^ORCID,Wang Leping¹^ORCID,Chen Quan³^ORCID,Tan Guangming¹²^ORCID

Affiliation:

1. State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China

2. University of Chinese Academy of Sciences, Beijing, China

3. Shanghai Jiao Tong University, Shanghai, China

Funder

NKRDP

NSFC

Beijing Nova Program

Youth Innovation Promotion Association of Chinese Academy of Sciences

CCF-Ant Research Fund CCF-AFSGRF

Pilotfor Major Scientific Research Facility of Jiangsu Province of China

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3625549.3658654

Reference55 articles.

1. The case for GPGPU spatial multitasking

2. AMD. Amd's approach to gpu virtualization. https://drivers.amd.com/relnotes/amd_mxgpu_deploymentguide_vmware.pdf.

3. AMD. Stream management hip api. https://docs.amd.com/bundle/HIP-API-Guide-v5.4.1/page/a00183.html.

4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901, 2020.

5. Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. {TVM}: An automated {End-to-End} optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 578--594, 2018.