Testing of statistical significance of semantic changes detected by diachronic word embedding

Abstract

In recent years, methods based on word embedding models have been widely used for solving problems of semantic change estimation. The models are trained on text corpora of various years. Semantic change is detected by analyzing changes in distance between words using vector space alignment or by analyzing changes in a set of words that are most similar in meaning to a target word. Testing for statistical significance of the detected effects has not been detailly discussed in previous studies. This paper focuses on the problem of testing statistical significance of semantic change. Besides, we consider the problem of finding a confidence interval of estimates of semantic distance between words. We allow for the influence of two random factors. The first one is associated with the use of random initial conditions and stochastic optimization when training the model, the second one results from a random selection of texts for a training corpus. The proposed approach is based on the use of resampling of a training set of texts. The proposed method is tested on the COHA corpus.