Using token-based semantic vector spaces for corpus-linguistic analyses: From practical applications to tests of theoretical claims

Abstract

AbstractThis paper presents token-based semantic vector spaces as a tool that can be applied in corpus-linguistic analyses such as word sense comparisons, comparisons of synonymous lexical items, and matching of concordance lines with a given text. We demonstrate how token-based semantic vector spaces are created, and we illustrate the kinds of result that can be obtained with this approach. Our main argument is that token-based semantic vector spaces are not only useful for practical corpus-linguistic applications but also for the investigation of theory-driven questions. We illustrate this point with a discussion of the asymmetric priming hypothesis (Jäger and Rosenbach 2008). The asymmetric priming hypothesis, which states that grammaticalizing constructions will be primed by their lexical sources but not vice versa, makes a number of empirically testable predictions. We operationalize and test these predictions, concluding that token-based semantic vector spaces yield conclusions that are relevant for linguistic theory-building.