Dealing with Design Failures in Randomized Field Experiments: Analytic Issues Regarding the Evaluation of Treatment Effects

Abstract

With an increasing number of criminal justice scholars conducting randomized field experiments, there are several analytic issues related to such studies that our discipline must begin to address more systematically. For example, treatment dilution and treatment migration are common forms of randomization implementation failure in field experiments, and a review of the criminological literature on experiments reveals a lack of consensus as to how these problems should be handled when evaluating treatment effects. In addition, issues related to statistical power and desired sample size remain unresolved. Given the relatively longer history of dealing with these issues in medicine, literature from that field is reviewed, providing additional insights regarding the dilemmas created by various design failures in randomized field experiments.