Affiliation:
1. Zhejiang University and Alibaba Group, China
2. Alibaba Group, China
3. Zhejiang University, China
Abstract
The rapid adoption of AI and convenience offered by cloud services have resulted in the growing demands for GPUs in the cloud. Generally, GPUs are physically attached to host servers as PCIe devices. However, the fixed assembly combination of host servers and GPUs is extremely inefficient in resource utilization, upgrade, and maintenance. Due to these issues, the GPU disaggregation technique has been proposed to decouple GPUs from host servers. It aggregates GPUs into a pool and allocates GPU node(s) according to user demands. However, existing GPU disaggregation systems have flaws in software-hardware compatibility, disaggregation scope, and capacity.
In this article, we present a new implementation of
datacenter-scale
GPU disaggregation, named DxPU. DxPU efficiently solves the above problems and can flexibly allocate as many GPU node(s) as users demand. To understand the performance overhead incurred by DxPU, we build up a performance model for AI specific workloads. With the guidance of modeling results, we develop a prototype system, which has been deployed into the datacenter of a leading cloud provider for a test run. We also conduct detailed experiments to evaluate the performance overhead caused by our system. The results show that the overhead of DxPU is less than 10%, compared with native GPU servers, in most of user scenarios.
Funder
National Key R&D Program of China
National Natural Science Foundation of China
Alibaba Group through Alibaba Research Intern Program
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture,Information Systems,Software
Reference51 articles.
1. 2014. PCI Express® Electrical Basics. Retrieved from https://pcisig.com/sites/default/files/files/PCI_Express_Electrical_Basics.pdf
2. 2018. Intel Rack Scale Design. Retrieved from https://www.kernel.org/doc/Documentation/ntb.txt
3. 2018. Intel Rack Scale Design Architecture. Retrieved from https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/rack-scale-design-architecture-white-paper.pdf
4. 2019. The Impact of Bit Errors in PCI Express® Links. Retrieved from https://www.asteralabs.com/insights/impact-of-bit-errors-in-pci-express-links/
5. 2019. PCI Express® 5.0 Architecture Channel Insertion Loss Budget. Retrieved from https://pcisig.com/pci-express%C2%AE-50-architecture-channel-insertion-loss-budget-0