The Effects of Embedding Closed-ended Cognitive Probes in a Web Survey on Survey Response

Abstract

Web, or online, probing has the potential to supplement existing questionnaire design processes by providing structured cognitive data on a wider sample than typical qualitative-only question evaluation methods can achieve. One of the practical impediments to the further integration of web probing is the concern of survey managers about how the probes themselves may affect response to other items and to a questionnaire as a whole. This study explores the effects web probes had on response to a self-administered web survey by comparing two rounds of this survey—one without web probes and one with web probes—that were administered to a probability-based panel of approximately 100,000 American adults. While the item response to the probes themselves appears to be related to the way they are formatted, the findings indicate that web probes do not have an overall negative effect on a questionnaire in which they are embedded.