Research on automatic classification of Chinese papers based on LDA model and TF-IDF algorithm

Abstract

Abstract To ensure the automatic classification of research directions in designated fields along with the natural semantic machine recognition of research papers, this research proposes an automatic classification method based on topic probability. Initially, the feature words and their topic probabilities are calculated through the LDA topic model, and the weight of feature words in each paper is calculated by means of the TF-IDF algorithm. Finally, the topic probabilities of each feature word in the paper are weighted and normalized, thus accomplishing the topic probability distribution from words to papers. Through unsupervised classification experiments on 53,780 papers in the field of clothing, the accuracy rate reaches 92.4%, and the F-score reaches 85.0% in comparison with professional manual classification. Consequently, the probability classification of paper topics proposed in this research can be directly used to solve the automatic classification of papers and the automatic classification of research directions.