Benchmark for Approximate Query Answering Systems

Abstract

The standard benchmark for Decision Support Systems is TPC-H, which is composed of a database, a workload, and a set of metrics for the performance evaluation. However, TPC-H does not include a methodology for the benchmark of Approximate Query Answering Systems, or the software tools used to obtain fast answers to analytical queries in the decision making process. In the paper, the authors present a methodology to evaluate and compare Approximate Query Answering Systems. To this aim, a methodology that extends the standard TPC-H and a set of new metrics that take into account the specific features of these systems are proposed. Experimental results show the application of these metrics to two systems based on the data analytic approximation by orthonormal series.