Distributed structural clustering on large graph

Abstract

SummaryGraph clustering is a primitive operation for graph data mining. It plays an important role to reveal community clusters, hubs, and outliers in complex networks. There are several graph clustering algorithms have been proposed based on the well‐studied SCAN algorithm in recent years. However, SCAN and its improved sequential variants are prohibitively slow due to their iterative computations. The parallel variants are focusing on improving the efficiency of graph clustering by utilizing multi‐cores computer architectures on single computing node with complex optimization techniques. Therefore, SCAN and its variants are not suitable for processing very large graphs due to the limitations of memory size and storage volume on a single node. In this article, we proposed a distributed parallel structural clustering algorithm using MapReduce. In order to improve the efficiency further, we proposed optimization techniques including partition based clustering and simplified combination with labels to accelerate the operations. We conducted extensive experiments on real world datasets. The experimental results showed our algorithm is high efficiency and scales well under different settings.