Acceptance of anomalous research findings: explaining treatment implausibility reduces belief in far-fetched results

Abstract

Research findings are best understood by considering contextual factors such as treatment plausibility: how likely it is that a studied treatment or manipulation is effective, based on theory and data. If a treatment is implausible, then more evidence should be required before believing it has an effect. The current study assessed the extent to which the interpretation of a research finding is affected by treatment plausibility. Participant age varied from 18 to 82 (M = 27.4, SD = 9.4), and about half of the participants (53%) were college students. A total of 600 participants read a brief news article about an experiment with a new type of psychotherapy for weight loss. The current study used a 2 (treatment plausibility) × 3 (results type) between-subjects factorial design. Treatment plausibility had two levels: (1) a plausible cognitive behavioral therapy and (2) an implausible ”psychic reinforcement therapy” that was described as employing psychic messages to promote weight loss. The three levels of the results type factor varied how the study results were presented in the article: (1) standard results with no mention of treatment plausibility, (2) standard results followed by interpretive statements focused on treatment plausibility, and (3) no results—the study was described as still in progress. Participants rated their belief in the effectiveness of the therapy on a scale of 0 to 100% in 10% increments. When treatment plausibility was not discussed in the article, average ratings for the implausible therapy were relatively high (M = 63.1%, SD = 25.0, 95% CI% [58.2–68.1]) and similar to those for the plausible therapy (M = 69.2%, SD = 21.5, 95% CI% [65.0–73.5]). Ratings for the implausible treatment were moderately lower when the article explained why the results supporting it were questionable (M = 48.5%, SD = 26.6, 95% CI% [43.2–53.8]). The findings of the current study suggest that students and other members of the public may draw incorrect inferences from research partly because they do not appreciate the importance of treatment plausibility. This could be remedied, though not completely, by explicitly discussing the plausibility of the treatment based on theory and prior data.