Using dual evolutionary search to construct decision tree based ensemble classifier

Abstract

AbstractA typical ensemble learning process typically uses a forward integration mechanism to construct the ensemble classifier with a large number of base classifiers. Based on this mechanism, it is difficult to adjust the diversity among base classifiers and optimize the structure inside ensemble since the generation process has a certain amount of randomness, which makes the performance of ensemble classifiers heavily dependent on the human design decisions. To address this issue, we proposed an automatic ensemble classifier construction method based on a dual-layer evolutionary search mechanism, which includes a tree coding-based base classifier population and a binary coding-based ensemble classifier population. Through a collaborative searching process between the two populations, the proposed method can be driven by training data to update the base classifier population and optimize the ensemble classifiers globally. To verify the effectiveness of the dual evolutionary ensemble learning method (DEEL), we tested it on 22 classification tasks from 4 data repositories. The results show that the proposed method can generate a diverse decision tree population on the training data while searching and constructing ensemble classifiers from them. Compared with 9 competitor algorithms, the proposed method achieved the best performance on 17 of 22 test tasks and improved the average accuracies by 0.97–7.65% over the second place. In particular, the generated ensemble classifiers show excellent structure, which involve small number and diverse decision trees. That increases the transparency of ensembles and helps to perform interpretability analysis on them.