SpotServe: Serving Generative Large Language Models on Preemptible Instances-Reference-Cited by-同舟云学术

SpotServe: Serving Generative Large Language Models on Preemptible Instances

Published:2024-04-27 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2
language:
Short-container-title:

Author:

Miao Xupeng¹^ORCID,Shi Chunan²^ORCID,Duan Jiangfei³^ORCID,Xi Xiaoli⁴^ORCID,Lin Dahua⁵^ORCID,Cui Bin²^ORCID,Jia Zhihao⁶^ORCID

Affiliation:

1. Carnegie Mellon University, Pittsburgh, PA, USA

2. Peking University, Beijing, China

3. The Chinese University of Hong Kong, Hong Kong, China

4. Carnegie Mellon University, Pittsburgh, USA

5. The Chinese University of Hong Kong & Sensetime Research, Hong Kong, China

6. Carnegie Mellon University, Pittsburgh, United States of America

Funder

NSF awards CNS

Amazon Research Award

Cisco Research Award

Google Faculty Research Award

Meta Research Award

Oracle Research Award

Qualcomm Innovation Fellowship

Samsung GRO Research Award

Publisher

ACM

Link

Reference61 articles.

1. Amazon EC2 Spot Instances. https://aws.amazon.com/ec2/spot/.

2. NVIDIA Triton Inference Server. https://developer.nvidia.com/nvidiatriton-inference-server.

3. Use Azure Spot Virtual Machines. https://learn.microsoft.com/en-us/azure/virtual-machines/spot-vms.

4. CUDA IPC. https://docs.nvidia.com/cuda/cuda-runtime-api/group_CUDART_DEVICE.html, 2021.

5. NVIDIA FasterTransformer. https://github.com/NVIDIA/FasterTransformer, 2021.

Cited by 2 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. QUART: Latency-Aware FaaS System for Pipelining Large Model Inference;2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS);2024-07-23

2. Demystifying Data Management for Large Language Models;Companion of the 2024 International Conference on Management of Data;2024-06-09