POSTER: FineCo: Fine-grained Heterogeneous Resource Management for Concurrent DNN Inferences-Reference-Cited by-同舟云学术

POSTER: FineCo: Fine-grained Heterogeneous Resource Management for Concurrent DNN Inferences

Published:2024-02-20 Issue: Volume:4 Page:451-453
ISSN:
Container-title:Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming
language:
Short-container-title:

Author:

Ma Lixian¹²^ORCID,Chen Haoruo¹²^ORCID,Shao En¹²³^ORCID,Wang Leping¹^ORCID,Chen Quan⁴^ORCID,Tan Guangming¹²^ORCID

Affiliation:

1. State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China

2. University of Chinese Academy of Sciences, Beijing, China

3. Nanjing Institute of InforSuperBahn, Nanjing, China

4. Shanghai Jiao Tong University, Shanghai, China

Funder

NKRDP

NSFC

Beijing Nova Program

Youth Innovation Promotion Association of Chinese Academy of Sciences

CCF-Ant Research Fund CCF-AFSGRF

Pilotfor Major Scientific Research Facility of Jiangsu Province of China

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3627535.3638485

Reference23 articles.

1. The case for GPGPU spatial multitasking

2. AMD. Amd's approach to gpu virtualization. https://drivers.amd.com/relnotes/amd_mxgpu_deploymentguide_vmware.pdf.

3. AMD. Stream management hip api. https://docs.amd.com/bundle/HIP-API-Guide-v5.4.1/page/a00183.html.

4. Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. {TVM}: An automated {End-to-End} optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 578--594, 2018.

5. KRISP: Enabling Kernel-wise RIght-sizing for Spatial Partitioned GPU Inference Servers