Open Questions Self-Administered on the Web versus Interviewer-Administered in Person: The 2016 American National Election Study Mode Comparison

Abstract

Abstract Relatively little is known about the performance of open questions in self-administered questionnaires compared to interviewer-administered ones. We examined this issue using the responses to the 2016 American National Election Study’s Web self-administered questionnaires and face-to-face computer-assisted personal interviews (CAPI) from independently drawn samples that asked three sets of open questions: most important problem(s) facing the country; candidate for and against reasons; and political party likes and dislikes. Although Web respondents gave shorter answers with fewer codeable ideas than face-to-face respondents, the two modes produced similar distributions of substantive answers. Despite our expectation that open answers to interviewers would be more civil than those expressed on the Web, there was no difference in civility between the modes. However, the construct validity of the four candidate for and against items (predicting vote choice of Clinton vs. Trump) was higher on the Web than in the interviewer-administered CAPI. These findings call into question the conventional wisdom that open questions are not well suited for self-administration and suggest that researchers should consider more frequent use of open questions in Web surveys.