Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations

Abstract

Background: In randomized controlled trials, attrition rates often differ by treatment status, jeopardizing causal inference. Inverse probability weighting methods and estimation of treatment effect bounds have been used to adjust for this bias. Objectives: We compare the performance of various methods within two samples, both generated through lottery-based randomization: one with considerable differential attrition and an augmented dataset with less problematic attrition. Research Design: We assess the performance of various correction methods within the dataset with problematic attrition. In addition, we conduct simulation analyses. Results: Within the more problematic dataset, we find the correction methods often performed poorly. Simulation analyses indicate that deviations from the underlying assumptions for bounding approaches damage the performance of estimated bounds. Conclusions: We recommend the verification of the underlying assumptions in attrition correction methods whenever possible and, when verification is not possible, using these methods with caution.