Is One Study as Good as Three? College Graduates Seem to Think So, Even if They Took Statistics Classes

Abstract

When people interpret the outcome of a research study, do they consider other relevant information such as prior research? In the current study, 251 college graduates read a single brief fictitious news article. The article summarized the findings of a study that found positive results for a new drug. Three versions of the article varied the amount and type of previous research: (a) two prior studies that found the drug did not work, (b) no prior studies of the drug, or (c) two prior studies that found the drug had a positive effect. After reading the article, participants estimated the probability the drug is effective. Average estimates were similar for the three articles, even for participants who reported more statistics experience. Overall, just 4% of participants appeared to use prior research to make probability estimates—most seemed to focus on the latest study, while ignoring or discounting prior studies. Implications for statistics education and reporting are discussed.