A Hybrid Machine Learning Model with Cost-function Based Outlier Removal and Its Application on Credit Rating

Abstract

Abstract This paper establishes a methodology to build hybrid machine learning models, aiming to combine the power of different machine learning algorithms on different types of features and hypothesis. A generic cost-based outlier removal algorithm is introduced as a step of pre-process of training data. Using this methodology, an experimental hybrid machine learning model is implemented for a crediting problem, which combines three types of machine learning algorithms SVM, DT and LR. Different metrics of training time, recall, precision, F1 score, and AUC are evaluated and compared with the results from single model of single SVM, DT, and LR. The new hybrid models in general shows improvement in performance on multiple metrics. this study further demonstrated that one learning model may work well on certain type of variables but not as effectively on other variables. Well defined procedures can be followed to create hybrid models to combine the power of different learning models, which are superior in dealing with dataset with complicate feature selection. This methodology can be explored with other algorithms and applications.