The Effects of Grids on Web Surveys Completed with Mobile Devices

Abstract

Question grids are common on Web surveys, and studies show that grids can affect how respondents complete surveys. However, there is little research that investigates the effects of grids on Web surveys completed on mobile devices. In this article, we evaluate the effects of question grids on response quality and measurement error for surveys taken on phones or tablets. Our study draws on a probabilistic Web survey. The survey included an experiment in which respondents were assigned to one of three question format conditions: one large grid, two small grids, or single item per page. We analyze how question grids affect response times and nondifferentiation as well as explore the interaction effects between grids and devices. Reductions in time associated with question grids were greater for surveys completed on mobile devices as opposed to those completed on computers. Likewise, the increases in nondifferentiation associated with question grids were greater for surveys completed on mobile devices. We find that effects of question grids on responses in Web surveys can differ across devices, and so researchers should be cautious of using grids on Web surveys as more people opt to do surveys on phones or tablets.