A Comparison of the Quality of Cochrane Reviews and Systematic Reviews Published in Paper-Based Journals

Abstract

This study set out to compare Cochrane reviews and reviews published in paper-based journals. Two assessment tools were used to collect the data, a 23-itemchecklist developed by Sacks and a nine-itemscale developed by Oxman. Cochrane reviews were found to be better at reporting some items and paper-based reviews at reporting others. The overall quality was found to be low. This represents a serious situation because clinicians, health policy makers, and consumers are often told that systematic reviews represent “the best available evidence.” In the period since this study, the Cochrane Collaboration has taken steps to improve the quality of its reviews through, for example, more thorough prepublication refereeing, developments in the training and support offered to reviewers, and improvements in the system for postpublication peer review. In addition, the use of evidence-based criteria (i.e., the QUOROM statement) for reporting systematic reviews may help further to improve their quality.