How to Detect and Influence Looking Up Answers to Political Knowledge Questions in Web Surveys

Abstract

Abstract When answering political knowledge questions in web surveys, respondents can look up the correct answer on the Internet. This response behavior artificially inflates political knowledge scores that are supposed to measure fact-based information. In the present study, we address the existing knowledge gaps of previous research regarding looking up answers to political knowledge questions in web surveys. We conducted an experimental study based on the German Internet Panel, a large-scale population survey that uses a probability-based sample. Based on this experiment, we show that instructions help to reduce the number of lookups to knowledge questions in web surveys. We provide further evidence that looking up answers results in more correct answers to knowledge questions and, thus, in inflated political knowledge scores. Finally, our findings illustrate the challenges and benefits of using self-reported or paradata-based lookup measures as well as a combined measure that aims at utilizing both to detect lookups to political knowledge questions in web surveys.