A Design and a Model for Investigating the Heterogeneity of Context Effects in Public Opinion Surveys

Abstract

Context effects on survey response, caused by the unobserved interaction between beliefs stored in personal memory and triggers generated by the structure of the survey instrument, are a pervasive challenge to survey research. The authors argue that randomized survey experiments on representative samples, when paired with facilitative primes, can enable researchers to model selection into variable context effects, revealing heterogeneity at the population level. The value of the design, and its associated modeling strategy, is demonstrated by its ability to deepen the interpretation of a treatment effect of international competitiveness framing on long-used items drawn from the Phi Delta Kappa/Gallup Poll and the General Social Survey about the quality of schooling in the United States, confidence in the leaders running public education, and support for spending to improve schools.