Subjective Measures of Liberal Democracy

Abstract

Using democracy in empirical work requires accurate measurement. Yet, most policy and academic research presupposes the accuracy of available measures. This article explores judge-specific measurement errors in cross-national indicators of liberal democracy. The authors evaluate the magnitude of these errors in widely used measures of democracy and determine whether their results replicate during a 17-year period (1972 to 1988). Then, they examine the nature of these systematic errors, hypothesizing that three different processes—(a) the information available for rating, (b) the judges' processing of this information, and (c) the method by which a judge's processing decisions are translated into a rating—could create error. The authors find that for the 17-year period from 1972 to 1988, there is unambiguous evidence of judge-specific measurement errors, which are related to traits of the countries. In the conclusion, the authors discuss the implications for democracy research and for other subjective measures.