Abstract
Topic models are widely used to discover the latent representation of a set of documents. The two canonical models are latent Dirichlet allocation, and Gaussian latent Dirichlet allocation, where the former uses multinomial distributions over words, and the latter uses multivariate Gaussian distributions over pre-trained word embedding vectors as the latent topic representations, respectively. Compared with latent Dirichlet allocation, Gaussian latent Dirichlet allocation is limited in the sense that it does not capture the polysemy of a word such as “bank.” In this paper, we show that Gaussian latent Dirichlet allocation could recover the ability to capture polysemy by introducing a hierarchical structure in the set of topics that the model can use to represent a given document. Our Gaussian hierarchical latent Dirichlet allocation significantly improves polysemy detection compared with Gaussian-based models and provides more parsimonious topic representations compared with hierarchical latent Dirichlet allocation. Our extensive quantitative experiments show that our model also achieves better topic coherence and held-out document predictive accuracy over a wide range of corpus and word embedding vectors which significantly improves the capture of polysemy compared with GLDA and CGTM. Our model learns the underlying topic distribution and hierarchical structure among topics simultaneously, which can be further used to understand the correlation among topics. Moreover, the added flexibility of our model does not necessarily increase the time complexity compared with GLDA and CGTM, which makes our model a good competitor to GLDA.
Publisher
Public Library of Science (PLoS)
Reference38 articles.
1. Blei DM, Jordan MI, Griffiths TL, Tenenbaum JB. Hierarchical Topic Models and the Nested Chinese Restaurant Process. In: Proceedings of the 16th International Conference on Neural Information Processing Systems. NIPS’03. Cambridge, MA, USA: MIT Press; 2003. p. 17–24.
2. O’Neill J, Robin C, O’Brien L, Buitelaar P. An Analysis of Topic Modelling for Legislative Texts. In: ASAIL@ICAIL; 2016.
3. Wen S, Zhao Z, Yan H. Detecting Malicious Websites in Depth through Analyzing Topics and Web-Pages. In: Proceedings of the 2nd International Conference on Cryptography, Security and Privacy. ICCSP 2018. New York, NY, USA: Association for Computing Machinery; 2018. p. 128–133.
4. A topic modelling analysis of white papers in security token offerings: Which topic matters for funding?;P Bongini;Technological Forecasting and Social Change,2022
5. Obot N, O’Malley L, Nwogu I, Yu Q, Shi WS, Guo X. From Novice to Expert Narratives of Dermatological Disease. In: 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops); 2018. p. 131–136.