A New Hybrid Multiclass Approach Based on KNN and SVM

Abstract

Support vector machine (SVM) is a machine learning method widely used in solving binary data classification problems due to its performance. Nevertheless, in practical problems of classification, there are often cases of the presence of more than two classes of objects in the original dataset. The paper considers a solution to the problem of SVM multiclass with the aim to increase the data classification quality based on a new way of hybridisation between SVM and [Formula: see text]-nearest neighbour (KNN) algorithms. The first phase of the approach is called the filtering phase. At this level, the feature space is split into two classes separated by a hyperplane. In the next step called review, we generate a second hyperplane, then we calculate the distance between each test pattern and the second hyperplane in the feature space using e.g. the KNN function. The result of the two phases is three classes instead of two produced by the conventional SVM. For evaluation purposes, dataset experiments are conducted on seven benchmark datasets that have high dimensionality and large size. Numerical experiments show that the 3SVM approach can improve not only the accuracy compared to other multiclass SVM approaches, but also the precision, recall, and [Formula: see text]-score.