Replication Redux: The Reproducibility Crisis and the Case of Deworming

Abstract

Abstract In 2004, a landmark study showed that an inexpensive medication to treat parasitic worms could improve health and school attendance for millions of children in many developing countries. Eleven years later, a headline in The Guardian reported that this treatment, deworming, had been “debunked.” The pronouncement followed an effort to replicate and re-analyze the original study, as well as an update to a systematic review of the effects of deworming. This story made waves amidst discussion of a reproducibility crisis in some of the social sciences. In this paper, I explore what it means to “replicate” and “reanalyze” a study, both in general and in the specific case of deworming. I review the broader replication efforts in economics, then examine the key findings of the original deworming paper in light of the “replication,” “reanalysis,” and “systematic review.” I also discuss the nature of the link between this single paper's findings, other papers’ findings, and any policy recommendations about deworming. Through this example, I provide a perspective on the ways replication and reanalysis work, the strengths and weaknesses of systematic reviews, and whether there is, in fact, a reproducibility crisis in economics.