Framing Bias in the Interpretation of Quality Improvement Data: Evidence From an Experiment

Abstract

Background: A growing body of public management literature sheds light on potential shortcomings to quality improvement (QI) and performance management efforts. These challenges stem from heuristics individuals use when interpreting data. Evidence from studies of citizens suggests that individuals’ evaluation of data is influenced by the linguistic framing or context of that information and may bias the way they use such information for decision-making. This study extends prospect theory into the field of public health QI by utilizing an experimental design to test for equivalency framing effects on how public health professionals interpret common QI indicators. Methods: An experimental design utilizing randomly assigned survey vignettes is used to test for the influence of framing effects in the interpretation of QI data. The web-based survey assigned a national sample of 286 city and county health officers to a "positive frame" group or a "negative frame" group and measured perceptions of organizational performance. The majority of respondents self-report as organizational leadership. Results: Public health managers are indeed susceptible to these framing effects and to a similar degree as citizens. Specifically, they tend to interpret QI information presented in a "positive frame" as indicating a higher level of performance as the same underlying data presenting in a "negative frame." These results are statistically significant and pass robustness checks when regressed against control variables and alternative sources of information. Conclusion: This study helps identify potential areas of reform within the reporting aspects of QI systems. Specifically, there is a need to fully contextualize data when presenting even to subject matter experts to reduce the existence of bias when making decisions and introduce training in data presentation and basic numeracy prior to fully engaging in QI initiatives.