How to Balance the Load on Heterogeneous Clusters

Abstract

The problem of computing a large set of different tasks on a set of heterogeneous resources connected by a network is very common nowadays in very different environments and load balancing is indispensable for achieving high performance and high throughput in systems such as clusters. Cluster heterogeneity increases the difficulty of balancing the load across the system nodes and, although the relationship between heterogeneity and load balancing is difficult to describe analytically, in this paper different models and performance metrics are proposed to describe heterogeneous cluster behavior and to perform an exhaustive analysis of the effects of heterogeneity on load balancing algorithm performance. This analysis allows us to propose efficient solutions capable of dealing with heterogeneity for all the load balancing algorithm stages. Furthermore, a load balancing algorithm has been implemented following these solutions to demonstrate, with experimental results, its efficiency on real heterogeneous clusters.