Misunderstanding Margin of Error

Abstract

For years, scholars have documented the frequency to which the margin of error is provided in news reports of polls. What has not been systematically analyzed is whether the stories that report margin of error use this information correctly to interpret and present poll results. This study examines poll reporting on the network evening news during the 2000 presidential general election. Of the seventy-eight stories that reported findings from specific polls, forty-five (55 percent) included information about the sampling error. For the stories in which the accuracy of horse-race declarations could be assessed by the data used in the story, 47 percent were “inaccurate” because they misused the term statistical dead heat, claimed that one candidate was ahead when the results were within the margin of error (and not qualified as such), or said that the results were “outside of the margin of error” when they were not. These mistakes were most prevalent on NBC. Some of these inaccurate reports revealed a fundamental misunderstanding about what plus or minus means. As a result, changes in likely voters’ preferences were exaggerated during the general election campaign.