Abstract
AbstractThe analysis of text data using artificial intelligence and statistical methods has become increasingly important in recent years. One application is the automatic assignment of documents. For this purpose, a classification model is trained on the basis of historical data. If the structure of the texts to be classified changes over time, the quality of the classification will decrease. Change point detection algorithms can counteract this. Such algorithms automatically detect changes in the structure of the texts and indicate that the trained classification model has to be adapted. However, the undesired influence of the length of the document needs to be handled when modeling the text data. We present a multinomial change-point model detecting changes in text structures. The results are supported by simulation studies.
Funder
Bundesministerium für Bildung und Forschung
Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM
Publisher
Springer Science and Business Media LLC
Subject
Applied Mathematics,Clinical Psychology,Experimental and Cognitive Psychology,Analysis
Reference39 articles.
1. Adams RP, MacKay DJC (2007) Bayesian online changepoint detection. arXiv preprint arXiv:0710.3742
2. Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau RJ (2011) Sentiment analysis of twitter data. In: Proceedings of the Workshop on Language in Social Media (LSM 2011)
3. Aminikhanghahi S, Cook DJ (2017) A survey of methods for time series change point detection. Knowl Inf Syst 51(2):339–367
4. Barry D, Hartigan JA (1993) Product partition models for change point problems. J Am Stat Assoc 88(421):309–319
5. Batsidis A, Horváth L, Martín N, Pardo L, Zografos K (2013) Change-point detection in multinomial data using phi-divergence test statistics. J Multivar Anal 118:53–66
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献