Replication with Attention to Numerical Accuracy

Abstract

Numerical issues matter in statistical analysis. Small errors occur when numbers are translated from paper and pencil into the binary world of computers. Surprisingly, these errors may be propagated and magnified through binary calculations, eventually producing statistical estimates far from the truth. In this replication and extension article, we look at one method of verifying the accuracy of statistical estimates by running these same data and models on multiple statistical packages. We find that for two published articles, Nagler (1994, American Journal of Political Science 38:230-255) and Alvarez and Brehm (1995, American Journal of Political Science 39:1055-1089), results are dependent on the statistical package used. In the course of our replications, we uncover other pitfalls that may prevent accurate replication, and make recommendations to ensure the ability for future researchers to replicate results.