Hybrid Partitioning-Density Algorithm for K-Means Clustering of Distributed Data Utilizing OPTICS

Abstract

The authors present the first clustering algorithm for use with distributed data that is fast, reliable, and does not make any presumptions in terms of data distribution. The authors' algorithm constructs a global clustering model using small local models received from local clustering statistics. This approach outperforms the classical non-distributed approaches since it does not require downloading all of the data to the central processing unit. The authors' solution is a hybrid algorithm that uses the best partitioning and density-based approach. The proposed algorithm handles uneven data dispersion without a transfer overload of additional data. Experiments were carried out with large datasets and these showed that the proposed solution introduces no loss of quality compared to non-distributed approaches and can achieve even better results, approaching reference clustering. This is an excellent outcome, considering that the algorithm can only build a model from fragmented data where the communication cost between nodes is negligible.