System Benchmarking on Public Clouds

Abstract

There are several public cloud providers that provide service across different cloud models such as IaaS, PaaS, and SaaS. End users require an objective means to assess the performance of the services being offered by the various cloud providers. Benchmarks have typically been used to evaluate the performance of various systems and can play a vital role in assessing performance of the different public cloud platforms in a vendor neutral manner. Amazon's EC2 Service is one of the leading public cloud service providers and offers many different levels of service. The research in this chapter focuses on system level benchmarks and looks into evaluating the memory, CPU, and I/O performance of two different tiers of hardware offered through Amazon's EC2. Using three distinct types of system benchmarks, the performance of the micro spot instance and the M1 small instance are measured and compared. In order to examine the performance and scalability of the hardware, the virtual machines are set up in a cluster formation ranging from two to eight nodes. The results show that the scalability of the cloud is achieved by increasing resources when applicable. This chapter also looks at the economic model and other cloud services offered by Amazon's EC2, Microsoft's Azure, and Google's App Engine.