A novel method for anomaly detection using beta Hebbian learning and principal component analysis

Abstract

Abstract In this research work a novel two-step system for anomaly detection is presented and tested over several real datasets. In the first step the novel Exploratory Projection Pursuit, Beta Hebbian Learning algorithm, is applied over each dataset, either to reduce the dimensionality of the original dataset or to face nonlinear datasets by generating a new subspace of the original dataset with lower, or even higher, dimensionality selecting the right activation function. Finally, in the second step Principal Component Analysis anomaly detection is applied to the new subspace to detect the anomalies and improve its classification capabilities. This new approach has been tested over several different real datasets, in terms of number of variables, number of samples and number of anomalies. In almost all cases, the novel approach obtained better results in terms of area under the curve with similar standard deviation values. In case of computational cost, this improvement is only remarkable when complexity of the dataset in terms of number of variables is high.