Do you prefer Obamacare or the affordable care act? Simulating an informed public to improve survey measurement

Abstract

Abstract Researchers have long known that the terms used in questions posed to respondents shape the answers they give. Processes underlying these differences have generally been attributed to respondents’ interpretations of the questions (i.e., what do the terms lead them to focus on) and examined as a framing effect. Yet evidence that people often answer questions they do not fully understand means it is also possible that some of this difference may stem from respondents’ ability to parse what the questions are asking about (i.e., do they comprehend the question). In three online survey experiments, we manipulate wordings for policy attitude questions about the DREAM Act, Trump’s trade disputes, and the Affordable Care Act to assess how both question comprehension and interpretation depend on the wordings selected. We find that reported attitudes vary as a joint function of both processes. By comparing response patterns and simulating individuals’ “informed responses” across wordings, we can disentangle differences due to incomplete understandings (and misunderstandings) from those attributable to partisan interpretations. Evidence that individuals with greater topical knowledge better recognize the similarity of different wordings confirms that some of the wording effect is associated not just with framing, but also with comprehension.