Abstract
Distance or dissimilarity matrices are widely used in applications. We study the relationships between the eigenvalues of the distance matrices and outliers and show that outliers affect the pairwise distances and inflate the eigenvalues. We obtain the eigenvalues of a distance matrix that is affected by k outliers and compare them to the eigenvalues of a distance matrix with a constant structure. We show a discrepancy in the sizes of the eigenvalues of a distance matrix that is contaminated with outliers, present an algorithm and offer a new outlier detection method based on the eigenvalues of the distance matrix. We compare the new distance-based outlier technique with several existing methods under five distributions. The methods are applied to a study of public utility companies and gene expression data.
Subject
Artificial Intelligence,Computer Vision and Pattern Recognition,Theoretical Computer Science
Reference24 articles.
1. U. Alon, N. Barkai, D.A. Notterman, K. Gish, S. Ybarra, D. Mack and A.J. Levine, Broad patterns of gene expression revealed by clustering of tumor and normal colon tissues probed by oligonucleotide arrays, Proc Natl Acad Sci USA 96(1) (1999), 6745–6750.
2. The Multivariate Skew-normal Distribution;Azzalini;Biometrika,1996
3. A. Azzalini, The Skew-Normal and Related Distributions Such as the Skew-t, Package “sn”, (2020), http//azzalini.stat.unipd.it/SN.
4. LOF: identifying density-based local outliers;Breunig;In ACM Sigmod Record,2000
5. Multivariate outlier detection based on a robust Mahalanobis distance with shrinkage estimators;Cabana;Statistical Papers,2021