Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems-Reference-Cited by-同舟云学术

Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systems

Published:2023-05-08 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 3rd Workshop on Machine Learning and Systems
language:
Short-container-title:

Author:

Salmani Mehran¹^ORCID,Ghafouri Saeid²^ORCID,Sanaee Alireza²^ORCID,Razavi Kamran³^ORCID,Mühlhäuser Max³^ORCID,Doyle Joseph²^ORCID,Jamshidi Pooyan⁴^ORCID,Sharifi Mohsen¹^ORCID

Affiliation:

1. Iran University of Science and Technology, Tehran, Iran

2. Queen Mary University of London, London, United Kingdom

3. Technical University of Darmstadt, Darmstadt, Germany

4. University of South Carolina, Columbia, USA

Publisher

ACM

Reference38 articles.

1. 2022. GPU virtualization in K8S: challenges and state of the art. https://www.arrikto.com/blog/gpu-virtualization-in-k8s-challenges-and-state-of-the-art/. (Nov 2022 ). 2022. GPU virtualization in K8S: challenges and state of the art. https://www.arrikto.com/blog/gpu-virtualization-in-k8s-challenges-and-state-of-the-art/. (Nov 2022).

2. 2022. Horizontal Pod autoscaling. (Jun 2022 ). https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ 2022. Horizontal Pod autoscaling. (Jun 2022). https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/

3. 2022. Inter-op parallelism threads. (2022). https://www.tensorflow.org/api_docs/python/tf/config/threading/set_inter_op_parallelism_threads 2022. Inter-op parallelism threads. (2022). https://www.tensorflow.org/api_docs/python/tf/config/threading/set_inter_op_parallelism_threads

4. 2022. Intra-op parallelism threads. (2022). https://www.tensorflow.org/api_docs/python/tf/config/threading/set_intra_op_parallelism_threads 2022. Intra-op parallelism threads. (2022). https://www.tensorflow.org/api_docs/python/tf/config/threading/set_intra_op_parallelism_threads

5. 2022. TorchServe parallelism threads. (2022). https://pytorch.org/docs/stable/notes/cpu_threading_torchscript_inference.html 2022. TorchServe parallelism threads. (2022). https://pytorch.org/docs/stable/notes/cpu_threading_torchscript_inference.html

Cited by 7 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Position Paper;Natural Language Processing Journal;2024-06

2. Resource allocation of industry 4.0 micro-service applications across serverless fog federation;Future Generation Computer Systems;2024-05

3. Sponge;Proceedings of the 4th Workshop on Machine Learning and Systems;2024-04-22

4. Enhancing Kubernetes Auto-Scaling: Leveraging Metrics for Improved Workload Performance;2023 Global Conference on Information Technologies and Communications (GCITC);2023-12-01

5. CAMEO;Proceedings of the 2023 ACM Symposium on Cloud Computing;2023-10-30