A Relief-PGS algorithm for feature selection and data classification

Abstract

As a supervised learning algorithm, Support Vector Machine (SVM) is very popularly used for classification. However, the traditional SVM is error-prone because of easy to fall into local optimal solution. To overcome the problem, a new SVM algorithm based on Relief algorithm and particle swarm optimization-genetic algorithm (Relief-PGS) is proposed for feature selection and data classification, where the penalty factor and kernel function of SVM and the extracted feature of Relief algorithm are encoded as the particles of particle swarm optimization-genetic algorithm (PSO-GA) and optimized by iteratively searching for optimal subset of features. To evaluate the quality of features, Relief algorithm is used to screen the feature set to reduce the irrelevant features and effectively select the feature subset from multiple attributes. The advantage of Relief-PGS algorithm is that it can optimize both feature subset selection and SVM parameters including the penalty factor and the kernel parameter simultaneously. Numerical experimental results indicated that the classification accuracy and efficiency of Relief-PGS are superior to those of other algorithms including traditional SVM, PSO-GA-SVM, Relief-SVM, ACO-SVM, etc.