An Experimental Evaluation of Alternative Methods for Case Prioritization in Responsive Survey Design

Abstract

Abstract Growing reluctance of households to participate in surveys has led to a variety of methodological efforts to combat this phenomenon. Several organizations employ case prioritization in a responsive survey design framework, dedicating increased effort to specific subgroups of sampled cases during certain data collection phases. For example, some surveys may prioritize subgroups defined by age and/or race/ethnicity if balanced response rates across these subgroups are important for minimizing nonresponse bias. Unfortunately, no methodological studies to date have identified optimal approaches for applying this increased effort to prioritized cases. This study experimentally examined three alternative methods for case prioritization in the National Survey of Family Growth: simply flagging the cases to receive increased effort in the sample management system (a “standard” method), developing tailored approaches to working the prioritized cases, and no prioritization (a “control” method). In the “tailored” method, which was designed to provide the interviewers with more guidance than simple “flagging,” interviewers worked with their supervisors to develop tailored strategies for how to best work each case within the prioritized subgroup. We find that both prioritization methods improved response rates and led to significant reductions in calling effort per completed case, with the “flagging” approach working particularly well. Given the additional costs associated with the “tailored” method, the results of our experiment provide support for a “hybrid” case prioritization approach that combines optimal features of these two methods.