The Impact of Mixing Survey Modes on Estimates of Change: A Quasi-Experimental Study

Abstract

Abstract Longitudinal surveys are a key data collection tool used to estimate social change. Recent developments have accelerated the move from traditional single-mode longitudinal designs to mixed-mode designs. Nevertheless, there are concerns that mixing survey modes may affect coefficients of change at the individual level. We investigate the impact of mixing survey modes on estimates of change using a quasi-experimental design implemented in a long-running UK panel study. Two types of comparisons are carried out: single-mode (face-to-face) design versus sequential mixed-mode (Web–face-to-face) design, and Web versus face to face. Across 41 variables, we find no differences in estimates of individual-level change across modes (designs). However, correlations between intercepts and slopes, an estimate of convergence of respondents, were significantly different for most variables, which led to some biases in estimates of change. Applied researchers are encouraged to do sensitivity checks to ensure their results are robust to mode effects.