Affiliation:
1. Google Inc.
2. University of Wisconsin, Madison
3. Microsoft Jim Gray Systems Lab
Abstract
Running parallel database systems in an environment with heterogeneous resources has become increasingly common, due to cluster evolution and increasing interest in moving applications into public clouds or shared infrastructures. For database systems running in a heterogeneous cluster, the default uniform data partitioning strategy may overload some of the slow machines while at the same time it may underutilize the more powerful machines. Since the processing time of a parallel query is determined by the slowest machine, such an allocation strategy may result in a significant query performance degradation.
We take a first step to address this problem by introducing a technique we call resource bricolage that improves database performance in heterogeneous environments. Our approach quantifies the performance differences among machines with various resources as they process workloads with diverse resource requirements. We formalize the problem of minimizing workload execution time and view it as an optimization problem, and then we employ linear programming to obtain a recommended data partitioning scheme. We verify the effectiveness of our technique with an extensive experimental study on a commercial database system.
Publisher
Association for Computing Machinery (ACM)
Subject
Information Systems,Software
Reference40 articles.
1. Aster Data nCluster. http://download.101com.com/tdwi/ww29/Aster Data A New Architecture.pdf. Aster Data nCluster. http://download.101com.com/tdwi/ww29/Aster Data A New Architecture.pdf.
2. Pivotal Greenplum. http://pivotal.io/big-data/pivotal-greenplum. Pivotal Greenplum. http://pivotal.io/big-data/pivotal-greenplum.
3. SQL Server 2012 Parallel Data Warehouse. http://www.microsoft.com/en-ca/servercloud/products/analytics-platform-system/. SQL Server 2012 Parallel Data Warehouse. http://www.microsoft.com/en-ca/servercloud/products/analytics-platform-system/.
4. Integrating vertical and horizontal partitioning into automated physical database design
5. Tarazu