Data-Centric Benchmarking

Abstract

In data management, both system designers and users casually resort to performance evaluation. Performance evaluation by experimentation on a real system is generally referred to as benchmarking. The aim of this chapter is to present an overview of the major past and present state-of-the-art data-centric benchmarks. This review includes the TPC standard benchmarks, but also alternative or more specialized benchmarks. Surveyed benchmarks are categorized into three families: transaction benchmarks aimed at On-Line Transaction Processing (OLTP), decision-support benchmarks aimed at On-Line Analysis Processing (OLAP) and big data benchmarks. Issues, tradeoffs and future trends in data-centric benchmarking are also discussed.