Same Question, Different Answers?

Abstract

Abstract: Psychological scientists increasingly study web data, such as user ratings or social media postings. However, whether research relying on such web data leads to the same conclusions as research based on traditional data is largely unknown. To test this, we (re)analyzed three data sets, thereby comparing web data with laboratory and online survey data. We calculated correlations across these different data sets (Study 1) and investigated identical, illustrative research questions in each data set (Studies 2–4). Our results suggest that web and traditional data are not fundamentally different and usually lead to similar conclusions, but also that it is important to consider differences between data types such as populations and research settings. Web data can be a valuable tool for psychologists when accounting for such differences, as it allows for testing established research findings in new contexts, complementing them with insights from novel data sources.