Microblog topic detection using evolutionary clustering and social network information

Abstract

SummaryThe messages in microblogs, like Twitter, present a wealth of information around the hot topics and public opinions. Topic detection and tracking enables a better understanding of the incoming information from social networks. Many of the studies performed in this field consider a set of predefined number of topics (clusters) which remain constant during the detection process. Such approaches are not efficient when data is dynamic and cumulative. In addition, nonparametric evolutionary topic models do not have an accurate performance on short texts due to the data deficiency. In this article, we propose an evolutionary clustering model based on the Bayesian nonparametric Dirichlet process mixture model, which automatically determines the number of clusters. To improve the algorithm performance when dealing with short social media texts, temporal information of tweets and other social network parameters are used in a weighted combination to augment the text similarity in detecting topics. Model evaluation is performed on the data gathered from the Twitter during a 2.5‐month period. Results reveal that the social network information can promote the topic detection performance. The proposed model outperforms previous study in terms of topic coherence and clustering performance, leading to a better clustering on short texts.