Tabi: An Efficient Multi-Level Inference System for Large Language Models-Reference-Cited by-同舟云学术

Tabi: An Efficient Multi-Level Inference System for Large Language Models

Published:2023-05-08 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the Eighteenth European Conference on Computer Systems
language:
Short-container-title:

Author:

Wang Yiding¹^ORCID,Chen Kai¹^ORCID,Tan Haisheng²^ORCID,Guo Kun³^ORCID

Affiliation:

1. Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong

2. University of Science and Technology of China, Hefei, China

3. Fuzhou University, Fuzhou, China

Publisher

ACM

Link

Reference89 articles.

1. GlusterFS. https://www.gluster.org/. GlusterFS. https://www.gluster.org/.

2. HuggingFace models. https://huggingface.co/models. HuggingFace models. https://huggingface.co/models.

3. HuggingFace Tokenizers. https://github.com/huggingface/tokenizers/. HuggingFace Tokenizers. https://github.com/huggingface/tokenizers/.

4. HuggingFace Transformers. https://github.com/huggingface/transformers/. HuggingFace Transformers. https://github.com/huggingface/transformers/.

5. PyTorch. https://pytorch.org/. PyTorch. https://pytorch.org/.

Cited by 12 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Arlo: Serving Transformer-based Language Models with Dynamic Input Lengths;Proceedings of the 53rd International Conference on Parallel Processing;2024-08-12

2. LILAC: Log Parsing using LLMs with Adaptive Parsing Cache;Proceedings of the ACM on Software Engineering;2024-07-12

3. Hybrid SLM and LLM for Edge-Cloud Collaborative Inference;Proceedings of the Workshop on Edge and Mobile Foundation Models;2024-06-03

4. Loki: A System for Serving ML Inference Pipelines with Hardware and Accuracy Scaling;Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing;2024-06-03

5. OTAS: An Elastic Transformer Serving System via Token Adaptation;IEEE INFOCOM 2024 - IEEE Conference on Computer Communications;2024-05-20