Impact of Question Type and Question Order on Tobacco Prevalence Estimates in US Young Adults: A Randomized Experiment

Abstract

Abstract The purpose of this study was to identify the effects of question type, order, and inclusion of product images on tobacco use estimates in a national sample of young adults. Participants aged 18–34 years (N = 4,100) in the Truth Initiative Young Adult Cohort Study (2016) were randomized to one of five question types assessing ever and past 30-day use of tobacco products: (1) “select all that apply” list (checklist, CL); (2) breakout items for each product (B); (3) breakout + images (B + I); (4) CL and B; and (5) CL and B + I. The order of question type was randomly assigned in groups 4 and 5. Bivariate analyses estimated product-specific prevalence by question type/order. Ever cigarette and cigar use prevalence was higher and ever e-cigarette use was lower in B and B + I than in CL. Ever hookah use was higher in B + I than in CL. Past 30-day e-cigarette use was 8.3% higher and past 30-day smokeless use was 13.0% higher in B + I than in CL. In groups 4 and 5, higher prevalence of ever cigarette, cigar, hookah, and past 30-day smokeless use was observed when B was presented first. Question type, order, and inclusion of images affect prevalence estimates of tobacco use. Implications This study identifies the effects of question type, order, and inclusion of product images on tobacco use estimates in a national sample of young adults. Ever use and past 30-day use prevalence estimates of specific products were affected by respondents answering breakout items or breakout items with images compared with respondents answering a checklist of items in the survey. Current surveys that include a “select all that apply” list format may be underestimating ever and past 30-day prevalence estimates of tobacco products.