3E-LDA

Abstract

Linear discriminant analysis (LDA) is one of the important techniques for dimensionality reduction, machine learning, and pattern recognition. However, in many applications, applying the classical LDA often faces the following problems: (1) sensitivity to outliers, (2) absence of local geometric information, and (3) small sample size or matrix singularity that can result in weak robustness and efficiency. Although several researchers have attempted to address one or more of the problems, little work has been done to address all of them together to produce a more effective and efficient LDA algorithm. This article proposes 3E-LDA, an enhanced LDA algorithm, that deals with all three problems as an attempt to further improve LDA. It proposes to learn a weighted median rather than the mean of the samples to deal with (1), to embed both between-class and within-class local geometric information to deal with (2), and to calculate the projection vectors in the null space of the matrix to deal with (3). Experiments on six benchmark datasets show that these three enhancements enable 3E-LDA to markedly outperform state-of-the-art LDA baselines in both accuracy and efficiency.