How much do you need: a randomised experiment of whether readers can understand the key messages from summaries of Cochrane Reviews without reading the full review

Abstract

Objective We explored whether readers can understand key messages without having to read the full review, and if there were differences in understanding between various types of summary. Design A randomised experiment of review summaries which compared understanding of a key outcome. Participants Members of university staff ( n = 36). Setting Universities on the island of Ireland. Method The Cochrane Review chosen examines the health impacts of the use of electric fans during heat waves. Participants were asked their expectation of the effect these would have on mortality. They were then randomly assigned a summary of the review (i.e. abstract, plain language summary, podcast or podcast transcription) and asked to spend a short time reading/listening to the summary. After this they were again asked about the effects of electric fans on mortality and to indicate if they would want to read the full Review. Main outcome measure Correct identification of a key review outcome. Results Just over half (53%) of the participants identified its key message on mortality after engaging with their summary. The figures were 33% for the abstract group, 50% for both the plain language and transcript groups and 78% for the podcast group. Conclusions The differences between the groups were not statistically significant but suggest that the audio summary might improve knowledge transfer compared to written summaries. These findings should be explored further using a larger sample size and with other reviews.