On the Use of System-Level Benchmarks for Comparing Public Cloud Environments

Abstract

The proliferation of public cloud providers and services offered necessitate that end users have benchmarking-related information that help compare the properties of the cloud computing environment being provided. System-level benchmarks are used to measure the performance of overall system or subsystem. This chapter surveys the system-level benchmarks that are used for traditional computing environments that can also be used to compare cloud computing environments. Amazon's EC2 Service is one of the leading public cloud service providers and offers many different levels of service. The research in this chapter focuses on system-level benchmarks and looks into evaluating the memory, CPU, and I/O performance of two different tiers of hardware offered through Amazon's EC2. Using three distinct types of system benchmarks, the performance of the micro spot instance and the M1 small instance are measured and compared. In order to examine the performance and scalability of the hardware, the virtual machines are set up in a cluster formation ranging from two to eight nodes.