Blending Probability and Nonprobability Samples with Applications to a Survey of Military Caregivers

Abstract

Abstract Probability samples are the preferred method for providing inferences that are generalizable to a larger population. However, in many cases, this approach is unlikely to yield a sample size large enough to produce precise inferences. Our goal here is to improve the efficiency of inferences from a probability sample by combining (or blending) it with a nonprobability sample, which is (by itself) potentially fraught with selection biases that would compromise the generalizability of results. We develop novel methods of statistical weighting that may be used for this purpose. Specifically, we make a distinction between weights that can be used to make the two samples representative of the population individually (disjoint blending) and those that make only the combined sample representative (simultaneous blending). Our focus is on weights constructed using propensity scores, but consideration is also given to calibration weighting. We include simulation studies that, among other illustrations, show the gain in precision provided by the convenience sample is lower in circumstances where the outcome is strongly related to the auxiliary variables used to align the samples. Motivating the exposition is a survey of military caregivers; our interest is focused on unpaid caregivers of wounded, ill, or injured US servicemembers and veterans who served following September 11, 2001. Our work serves not only to illustrate the proper execution of blending but also to caution the reader with respect to its dangers, as invoking a nonprobability sample may not yield substantial improvements in precision when assumptions are valid and may induce biases in the event that they are not.