Affiliation:
1. GESIS – Leibniz Institute for the Social Sciences, Mannheim, Germany
Abstract
Probes are follow-ups to survey questions used to gain insights on respondents’ understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued for embedding more open-ended probes in large-scale web surveys. However, there are concerns that this may cause reactivity and impact survey data. The study presents a randomized experiment in which identical survey questions were run with and without open-ended probes. Embedding open-ended probes resulted in higher levels of survey break off, as well as increased backtracking and answer changes to previous questions. In most cases, there was no impact of open-ended probes on the cognitive processing of and response to survey questions. Implications for embedding open-ended probes into web surveys are discussed.
Funder
Deutsche Forschungsgemeinschaft
Subject
Sociology and Political Science,Social Sciences (miscellaneous)
Reference99 articles.
1. ADM, ASI, BVM and DGOF. 2021. “Richtlinie für Online-Befragungen.” (https://www.adm-ev.de/wp-content/uploads/2021/03/RL-Online-2021-neu.pdf).
2. null
3. Is the Left-Right Scale a Valid Measure of Ideology?