A MULTI-CLASS SUPPORT VECTOR MACHINE: THEORY AND MODEL

Abstract

A multi-class support vector machine (M-SVM) is developed, its dual is derived, its dual is mapped to high dimensional feature spaces using inner product kernels, and its performance is tested. The M-SVM is formulated as a quadratic programming model. Its dual, also a quadratic programming model, is very elegant and is easier to solve than the primal. The discriminant functions can be directly constructed from the dual solution. By using inner product kernels, the M-SVM can be built and nonlinear discriminant functions can be constructed in high dimensional feature spaces without carrying out the mappings from the input space to the feature spaces. The size of the dual, measured by the number of variables and constraints, is independent of the dimension of the input space and stays the same whether the M-SVM is built in the input space or in a feature space. Compared to other models published in the literature, this M-SVM is equally or more effective. An example is presented to demonstrate the dual formulation and solution in feature spaces. Very good results were obtained on benchmark test problems from the literature.