Correlation Versus Interchangeability: The Limited Robustness of Empirical Findings on Democracy Using Highly Correlated Data Sets

Abstract

This article shows that highly correlated measures can produce different results. We identify a democratization model from the literature and test it in more than 120 countries from 1951 to 1992. Then, we check whether the results are robust regarding measures of democracy, time periods, and levels of development. The findings show that measures do matter: Whereas some of the findings are robust, most of them are not. This explains, in part, why the debates on democracy have continued rather than been resolved. More important, it underscores the need for more careful use of measures and further testing to increase confidence in the findings. Scholars in comparative politics are drawn increasingly to large-N statistical analyses, often using data sets collected by others. As in any field, we show how they must be careful in choosing the most appropriate measures for their studies, without assuming that any correlated measure will do.