Is Bare-metal I/O Performance with User-defined Storage Drives Inside VMs Possible? Benchmarking libvfio-user vs. Common Storage Virtualization Configurations

Abstract

It is a critical concern for cloud computing providers to identify and adopt optimal virtual machine (VM) I/O storage paths. In Linux, these involve NVMe hardware, PCIe passthrough with vfio, and userspace drivers (e.g. SPDK). One project tying together these host-side improvements with the guests is vfio-user, a library and protocol for inter-process communication of I/O operations over sockets. QEMU/KVM VMs using vfio-user and SPDK are theoretically expected to attain low latencies due to mostly userspace processing, but this VM datapath configuration is still experimental and performance data is lacking in the literature.