Affiliation:
1. University of Nebraska-Lincoln, USA
Abstract
How to display questions that are part of a battery in self-administered surveys is an important decision. Battery items may be displayed in a grid in a mail survey or computer web survey, but are often displayed as individual items on mobile devices. Although past research has compared grids to item-by-item displays in computer and mobile web surveys, almost no work has compared these displays in mail surveys. Additionally, many web survey templates use wide rectangular buttons to select response options in individual items using a mobile-optimized design, different from the standard round answer space format typically used in mail surveys. In this study, we experimentally test grid versus item-by-item displays and round radio buttons versus wide rectangular buttons for battery items in a probability-based general population mixed-mode mail + web survey of adults in Nebraska. Consistent with past research, we find that item-by-item displays reduce straightlining rates compared to grid designs. We also find that respondents are less likely to select the last two response categories in the item-by-item displays than in the grid displays. Smartphone and computer web respondents have higher item nonresponse rates than mail respondents, and web respondents have lower straightlining rates than mail respondents, accounting for respondent characteristics. Reassuringly, there is no difference in data quality outcomes across radio button versus wide button formats. These findings replicate past research that item-by-item displays reduce straightlining but may shift answer categories. These findings suggest that questionnaire designers can combine round radio button answer spaces on mail surveys with wide buttons on web surveys on battery items with little difference in data quality.
Funder
UNL Office of Research and Economic Development
National Science Foundation National Center for Science and Engineering Statistics
Subject
Marketing,Economics and Econometrics,Business and International Management
Reference57 articles.
1. Effects of Mobile versus PC Web on Survey Response Quality
2. Using Buttons as Response Options in Mobile Web Surveys
3. Baker-Prewitt J., Miller J. (2013). Mobile research risk: What happens to data quality when respondents use a mobile device for a survey designed for a PC. CASRO online research conference. March 7-8, 2013. San Francisco, CA.
4. Callegaro M., Shand-Lubbers J., Dennis J. M. (2009). Presentation of a single item versus a grid: Effects on the vitality and mental health scales of the SF-36v2 health survey. Annual meeting of the American association for public opinion research. May 14-17, 2009. Hollywood, FL.