Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse

Abstract

Random digit dialed telephone surveys are facing two serious problems undermining probability-based inference and creating a potential for bias in survey estimates: declining response rates and declining coverage of the landline telephone frame. Optimum survey designs need to focus reduction techniques on errors that cannot be addressed through statistical adjustment. This requires (a) separating and estimating the relative magnitude of different error sources and (b) evaluating the degree to which each error source can be statistically adjusted. In this study, the authors found significant differences in means both for nonrespondents and for the eligible population excluded from the landline frame, which are also in opposite directions. Differences were also found for element variances and associations, which can affect survey results but are rarely examined. Adjustments were somewhat effective in decreasing both sources of bias, although addressing at least one through data collection led to less bias in the adjusted estimates.