Enhancing Clustering Performance Using Topic Modeling-Based Dimensionality Reduction

Abstract

Mainly in the present times, the description of the services and their working procedure have been established in natural text language. We have obtained service groups based on their similarities to reduce search space and time in service innovation. Major topic models such as LSA, LDA, and CTM policies have not been able to show effective performance due to the short description and limited description of services in text form, the reduction or absence of words that occur. To solve the issues created by brief text, the Dirichlet Multinomial Mixer model (DMM) with features representation using the Gibbs algorithm has been developed to reduce dimensionality in clustering and enhance performance. The launch results prove that DMM-Gibbs can give better results than all other methods with agglomerative or K-means clustering methods by sampling. Evaluations with internal and external criteria were used to calculate clustering performance based on these two values. Using this standard model, the dimensionality can be reduced to 93.13% and better clustering performance can be achieved.