Affiliation:
1. School of Computing, Xi’an Shiyou University, Xi’an 710065, China
Abstract
Outlier detection is an important task in the field of data mining and a highly active area of research in machine learning. In industrial automation, datasets are often high-dimensional, meaning an effort to study all dimensions directly leads to data sparsity, thus causing outliers to be masked by noise effects in high-dimensional spaces. The “curse of dimensionality” phenomenon renders many conventional outlier detection methods ineffective. This paper proposes a new outlier detection algorithm called EOEH (Ensemble Outlier Detection Method Based on Information Entropy-Weighted Subspaces for High-Dimensional Data). First, random secondary subsampling is performed on the data, and detectors are run on various small-scale sub-samples to provide diverse detection results. Results are then aggregated to reduce the global variance and enhance the robustness of the algorithm. Subsequently, information entropy is utilized to construct a dimension-space weighting method that can discern the influential factors within different dimensional spaces. This method generates weighted subspaces and dimensions for data objects, reducing the impact of noise created by high-dimensional data and improving high-dimensional data detection performance. Finally, this study offers a design for a new high-precision local outlier factor (HPLOF) detector that amplifies the differentiation between normal and outlier data, thereby improving the detection performance of the algorithm. The feasibility of this algorithm is validated through experiments that used both simulated and UCI datasets. In comparison to popular outlier detection algorithms, our algorithm demonstrates a superior detection performance and runtime efficiency. Compared with the current popular, common algorithms, the EOEH algorithm improves the detection performance by 6% on average. In terms of running time for high-dimensional data, EOEH is 20% faster than the current popular algorithms.
Funder
Shaanxi Key Laboratory for Network Computing and Security Technology
Subject
General Physics and Astronomy
Reference40 articles.
1. Outlier detection: Methods, models, and classification;Boukerche;ACM Comput. Surv. CSUR,2020
2. Goldstein, M. (2023). Special Issue on Unsupervised Anomaly Detection. Appl. Sci., 13.
3. Outlier Detection Algorithms for Open Environments;Kou;Wirel. Commun. Mob. Comput.,2023
4. Zhang, J., and Yang, Y. (2023). Density-Distance Outlier Detection Algorithm Based on Natural Neighborhood. Axioms, 12.
5. Fuzzy information entropy-based adaptive approach for hybrid feature outlier detection;Yuan;Fuzzy Sets Syst.,2021
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献