Survey Nonresponse and Mass Polarization: The Consequences of Declining Contact and Cooperation Rates

Abstract

Recent studies question whether declining response rates in survey data overstate the level of polarization of Americans. At issue are the sources of declining response rates—declining contact rates, associated mostly with random polling mechanisms, or declining cooperation rates, associated with personal preferences, knowledge, and interest in politics—and their differing effects on measures of polarization. Assessing 158 surveys (2004–2018), we show that declining cooperation is the primary source of declining response rates and that it leads to survey overrepresentation of people who are more engaged in politics. Analyzing individual responses to 1,223 policy questions in those surveys, we further show that, conditional on the policy area, this survey bias overestimates or underestimates the partisan divide among Americans. Our findings question the perceived strength of mass polarization and move forward the discussion about the effect of declining survey response on generalizations from survey data.