Big Data Analytics Model for Distributed Document Using Hybrid Optimization with K -Means Clustering

Abstract

Clustering, also known as unsupervised learning, is one of the most significant topics of machine learning because it divides data into groups based on similarity with the aim of extracting or summarizing new information. It is one of the most often used machine learning techniques. The most significant problem encountered in this subject is the sheer volume of electronic text documents accessible, which is increasing at an exponential rate, necessitating the development of efficient ways for dealing with these papers. Furthermore, it is not practicable to consolidate all of the papers from numerous locations into a single area for processing. In this study, the primary goal is to enhance the performance of the distributed document clustering approach for clustering big, high-dimensional distributed document datasets. For distributed storage and analysis, one of the most prominent open-source implementations of the big data analytic-based MapReduce model, such as the Hadoop framework, is used in conjunction with a distributed file system and is known as the Hadoop Distributed File System, to achieve the desired results. This necessitates an improvement in the approach of the clustering operation with Elephant Herding Optimization, which will be accomplished by applying a hybridized clustering procedure. In conjunction with the MapReduce framework, this hybridized strategy is able to solve the obstacles associated with the $K$ -means clustering method, including the initial centroids difficulty and the dimensionality problem. In this paper, we analyze the performance of the whole distributed document clustering technique for big document datasets by using a distributed document clustering framework such as Hadoop and the associated MapReduce methodology. In the end, this decides how quickly computations may be completed.