Ensemble Self-Paced Learning Based on Adaptive Mixture Weighting

Abstract

Self-paced learning (SPL) is a learning mechanism inspired by human and animal learning processes that gives variable weights to samples, gradually introducing simple to more complicated samples into the learning set as the “age” parameter increases. To regulate the learning process, a self-paced weighting regularization term with an “age” parameter is introduced to the learning function. Several self-paced weighting methods have been proposed, and different regularization terms might result in varied learning performance. However, on the one hand, it is difficult to select a suitable weighting method for SPL. On the other hand, it is challenging to determine the “age” parameter, and it is easy for SPL to obtain poor results as the “age” of the model increases. To solve the aforementioned difficulties, an ensemble SPL approach with an adaptive mixture weighting mechanism is proposed in this study. First, as the “age” parameter increases, a set of base classifiers is collected to produce a new data set, which is used to learn the second-level classifier. Then, the ensemble model is used to generate the final output to avoid the selection of the optimal “age” parameter. An adaptive mixture weighting method is designed to reduce the dependence of parameters on human experience. The previous methods find it difficult to determine the “age” parameters or self-paced parameters. In this paper, these parameters can be adjusted adaptively during the learning process. In comparison with the previous SPL techniques, the proposed method achieves the best results in 27 of the 32 datasets in the experiments with the adaptive parameters. The statistical tests are carried out to show that the proposed method is superior to other state-of-the-art algorithms.