A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer

Abstract

ObjectiveTo compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.DesignMethodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.Data sourcesCancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.Eligibility criteria for selecting studiesWe included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.ResultsWe included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95% CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95% CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95% CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95% CI 0.02 to 0.09), use individual patient data (RR 0.03; 95% CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95% CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0–143); high-impact journal SRs: 74.45 (0–652)).ConclusionsWhen comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.