Concurrent, Web-First, or Web-Only? How Different Mode Sequences Perform in Recruiting Participants for a Self-Administered Mixed-mode Panel Study

Abstract

Abstract During the COVID-19 pandemic, many survey programs switched to self-administered modes of data collection, often offering respondents both web-based and paper-based questionnaires. However, there is little guidance as to when to offer which mode, especially when the aim is to recruit participants for a panel survey. This study examines the effectiveness of different mode-sequence designs by analyzing an experiment implemented in the recruitment wave of the German panel study “Family Research and Demographic Analysis.” We randomly assigned 108,256 individuals aged 18–49 years to one of three mode-sequence-design conditions: concurrent, web-first including a paper-based questionnaire with the second reminder (web-first-2), and web-first including a paper-based questionnaire with the third reminder (web-first-3). A fourth, simulated group did not receive a paper-based questionnaire (web-only). We analyzed how different mode-sequence designs affected outcome rates, sample composition, response distributions, data quality, share of paper-based questionnaires, postage costs, number of postal mailings in the recruitment wave, and participation in the first regular wave. Our results show no differences in response distributions and small differences in sample composition across the four mode-sequence designs. As the web-first-2 and simulated web-only designs yielded comparatively good response and recruitment rates at reasonable costs, we recommend their use when surveying adults between 18 and 49 years old.