Using Visual Grouping to Improve Navigation of Skip Patterns in Mail Surveys: an Experiment

Abstract

Abstract Skip patterns, in which certain questions should only be answered by subsets of respondents, are particularly challenging for mail surveys because respondents must navigate them themselves with no assistance from a computer or interviewer. Incorrect navigation of skip patterns can lead to (1) errors of omission, which is when respondents fail to answer follow-up questions they should answer; (2) errors of commission, which is when respondents answer follow-up questions they should skip; or (3) filter item nonresponse, which is when respondents fail to answer the initial filter question. Using experimental data from two general population surveys of Nebraskans, this paper examines three visual design elements—common region, indentation, and subnumbering—aimed at decreasing skip errors in mail surveys. Each design element is intended to create stronger grouping and subgrouping among items within skip patterns, thus clarifying the navigational path and decreasing skip errors. We compare error rates across treatments overall and by respondent age, education, and literacy to assess whether visual cues can provide additional aid to those who might struggle with reading text cues. We find that using separate enclosures or subnumbering of the follow-up questions significantly reduces commission error rates and using indentation significantly increases omission error rates. In addition, none of the tested visual designs were more effective for older, less educated, or lower literacy respondents than for their counterparts.