Assessing the Validity of Enns and Koch's Measure of State Policy Mood

Abstract

Enns and Koch (hereafter E&K) use multilevel regression and poststratification (MRP) along with survey aggregation to measure state policy mood. As E&K rely on direct information about public opinion, it would be preferable to Berry et al.‘s widely used indirect measure relying on data about the issue positions and vote shares of members of Congress, if E&K's measure were valid. Assessing the validity of E&K's measure takes on special importance because the measure proves to be nearly uncorrelated with Berry et al.‘s measure, implying that at least one is invalid. Because the “true” policy mood of states is unknown, it is impossible to definitively assess the validity of E&K's measure. Instead, we raise some concerns about E&K's measurement methodology and present evidence pertaining to the indicator's face validity, convergent validity, and construct validity. Our analyses leave us doubtful that the E&K measure is valid because its characterization of state moods departs significantly from conventional wisdom and current scholarship.