WattWiser: Power & Resource-Efficient Scheduling for Multi-Model Multi-GPU Inference Servers-Reference-Cited by-同舟云学术

WattWiser: Power & Resource-Efficient Scheduling for Multi-Model Multi-GPU Inference Servers

Published:2023-10-28 Issue: Volume: Page:39-44
ISSN:
Container-title:Proceedings of the 14th International Green and Sustainable Computing Conference
language:
Short-container-title:

Author:

Jahanshahi Ali¹^ORCID,Rezvani Mohammadreza¹^ORCID,Wong Daniel²^ORCID

Affiliation:

1. Department of Computer Science & Engineering, University of California, Riverside, USA

2. Department of Electrical & Computer Engineering, University of California, Riverside, USA

Funder

NSF (National Science Foundation)

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3634769.3634807

Reference11 articles.

1. Seungbeom Choi, Sunho Lee, Yeonjae Kim, Jongse Park, Youngjin Kwon, and Jaehyuk Huh. 2021. Multi-model machine learning inference serving with gpu spatial partitioning. arXiv preprint arXiv:2109.01611 (2021).

2. Seungbeom Choi, Sunho Lee, Yeonjae Kim, Jongse Park, Youngjin Kwon, and Jaehyuk Huh. 2022. Serving Heterogeneous Machine Learning Models on Multi-GPU Servers with Spatio-Temporal Sharing. In 2022 USENIX Annual Technical Conference (USENIX ATC 22). 199–216.

3. Marcus Chow, Ali Jahanshahi, and Daniel Wong. 2023. KRISP: Enabling Kernel-wise Right-sizing for Spatial Partitioned GPU Inference Servers. In 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE.

4. GSLICE

5. Pieter Hintjens. 2013. ZeroMQ: messaging for many applications. " O’Reilly Media, Inc.".

Cited by 1 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Characterizing In-Kernel Observability of Latency-Sensitive Request-Level Metrics with eBPF;2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS);2024-05-05