Detecting Link Communities Based on Hadoop

Abstract

Overlapping community detection is a key approach to study the structure of network. Link community approach redefines the community into the collection of edges, which shows great advantages. But with the increasing number of edges in the network, the efficiency of link community algorithm is severely degraded, so that this algorithm lost scalability. In order to detect the large overlapping community efficiently, we proposed a method DLCH (detecting link communities based on Hadoop), mapping link communities method to MapReduce framework, an efficient detecting communities platform for large complex communities. It can successfully store and process massive data by HDFS, and divide the work to map and reduce tasks, which can detect the complex communities efficiently. According to the experiments on standard test data, this method is scalable and effective.