On the Performance Evaluation of IaaS Cloud Services With System-Level Benchmarks

Abstract

Infrastructure-as-a-service is a cloud service model that allows customers to outsource computing resources such as servers and storage. This article evaluates four IaaS cloud services - Amazon EC2, Microsoft Azure, Google Compute Engine and Rackspace Cloud in a vendor-neutral approach with regards to system parameter usage including server, file I/O and network utilization. Thus, system-level benchmarking provides objective comparison of cloud providers from performance standpoint. Unixbench, Dbench and Iperf are the System-level benchmarks chosen to test the performance of server, file I/O and network respectively. In order to capture the variation in performance, the tests were performed at different times on weekdays and weekends. With each offering, the benchmarks are tested on different configurations to provide an insight to the cloud users in selection of provider followed by appropriate VM sizing according to the workload requirement. In addition to the performance evaluation, price-per-performance value of all the providers is also examined and compared.