Independence and generalizability in linguistics

Abstract

Abstract Quantitative studies in linguistics almost always involve data points that are related to each other, such as multiple data points from the same participant, multiple texts from the same book, author, genre, or register, or multiple languages from the same language family. Statistical procedures that fail to account for the relatedness of observations by assuming independence among units can lead to grossly misleading results if these sources of variation are ignored. As mixed effects models are increasingly used to analyze these non-independent data structures, it might appear that the problem of violating the independence assumption is solved. In this paper, we argue that it is necessary to re-open and widen the discussion about sources of variation that are being ignored, not only in statistical analyses, but also in the way studies are designed. Non-independence is not something that is “solved” by new statistical methods such as mixed models, but it is something that we continuously need to discuss as we apply new methods to an increasingly diverse range of linguistic datasets and corpora. In addition, our paper delivers something that is currently missing from statistical textbooks for linguists, which is an overview of non-independent data structures across different subfields of linguistics (corpus linguistics, typology, phonetics etc.), and how mixed models are used to deal with these structures.