Benefits of Adaptive Design Under Suboptimal Scenarios: A Simulation Study

Abstract

Abstract Theories have suggested that adaptive survey designs may reduce biases and variances of survey estimates more so than post-survey adjustment. However, previous research has not incorporated practical constraints into theorizing the utility of adaptive design. How would design factors and suboptimal auxiliary information influence the effect of adaptive design? The current simulation considers three separate components in adaptive design: (1) Modeling—how sample cases’ response propensities are predicted by a response propensity model; (2) Operationalization—how researchers decide to differentially allocate recruitment effort to sample cases, and (3) Achievement—the sizes of the changes in response propensities that are achieved by the adaptive strategies. Each component influences the effect of adaptive design on the biases and variances of survey estimates. The simulation study presented here suggests that adaptive designs based on adequate auxiliary information improve survey estimates more so than post-survey adjustment. However, adaptive designs may backfire if the strategies are not correctly designed due to a lack of critical auxiliary variables.