How do Systematic Reviews of Acupuncture for Pain Relief Incorporate Risk of Bias Assessments into the Synthesis? A Methodological Study

Abstract

Objective By reviewing the assessment of internal validity in relevant systematic reviews (SRs), the aim of this study was to identify how critical appraisals of risk of bias (RoB) inform the synthesis of evidence in SRs of acupuncture for pain relief. Methods SRs were searched in Medline, EMBASE, and the Cochrane Database of SRs from their inception to 30 December 2014. Only SRs of acupuncture for pain relief were included. Basic information, types of RoB appraisal tool, whether or not there was domain-level assessment of RoB, whether or not the reviews ranked studies by RoB, plus whether or not (and, if so, how) RoB appraisal was incorporated into the synthesis were determined. Results A total of 91 SRs met the inclusion criteria and were included in the final analysis. Over half of the SRs (85, 64.8%) used standard tools, such as the Jadad quality score and the Cochrane RoB tool, followed by adapted tools (n=23, 25.3%). Of the 85 SRs that assessed RoB, 29 (34.1%) presented domain-level assessment and 71 SRs (83.5%) included ranking of the studies based on RoB assessment. Of these 71, 35 (49.4%) used a cut-off threshold score and 26 (36.6%) required all criteria sum-up. Of the 85 SRs that assessed RoB, 48 (56.5%) incorporated RoB appraisal into the data synthesis. Conclusions Although most SRs of acupuncture for pain relief conducted some form of RoB assessment, nearly half of them failed to incorporate the RoB assessment into the synthesis.