A Machine Learning based Resource Efficient Task Scheduler for Heterogeneous Computer Systems

Abstract

Abstract Heterogeneous computer systems are becoming mainstream due to disparate processing and performance capabilities of multi-core architectures. It consists of different type of devices, i.e., Central Processing Units (CPUs), accelerators, and Graphics Processing Units (GPUs). In the heterogeneous computing environment, if one device is more powerful in terms of computing capability, the scheduling schemes generally favor the powerful device, and that device becomes overloaded while the other device is underutilized. This load imbalance problem results in increased execution time. In this research, we propose load-balanced task scheduler combined with machine learning based device predictor. The device predictor is used to predict execution time both on CPU and GPU devices, and a device with shorter predicted execution time is considered as a suitable device for that particular task. However, it may happen that a high fraction of tasks map only on one type of device since that device is a suitable device for them. Such situation leads to the problem of load imbalance. We use work stealing based task scheduler as part of our solution that allows an idle device to process tasks from the queue of another’s device. In this way we can avoid load imbalance, minimize the overall execution time of tasks, and maximize the device utilization and throughput. We evaluate the performance of our proposed solution into two stages. Firstly, we measure the error rate of our machine learning predictor using three different algorithms (i.e., random forest, gradient boosting, and multiple linear regression). We demonstrate that random forest performs better with marginal error rate. Secondly, we compare the performance of work stealing task scheduler with other scheduling alternatives. Our results show that the proposed solution reduces execution time by 65.63%, increased resource utilization by 93.3%, and throughput by 65.5% in comparison to baseline scheduling schemes.