Methodological quality and risk of bias in orthodontic systematic reviews using AMSTAR and ROBIS

Abstract

Summary Background This study aimed to compare the methodological quality and risk bias of orthodontic systematic reviews (SRs) using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) and ROBIS (Risk of Bias in Systematic Review) tools. Materials and Methods A search of electronic databases (OVID and Medline) was undertaken to identify orthodontic SRs published in five primary orthodontic journals (January 2015 to December 2018) and the Cochrane Library of Systematic Reviews (January 2000 to January 2018). Full articles were reviewed by two assessors against the eligibility criteria. Methodological quality of each SR was gauged using the AMSTAR tool with a score of 0 or 1 given for each of the 11 items. Cumulative totals were calculated and scores between 4 and 8 represented poor to fair methodological quality and 9 or greater deemed to be good. As per the ROBIS tool, the risk of bias (ROB) for each domain was assessed and the overall ROB was classified as low, high, or unclear. Results A total of 91 SRs were included. The median AMSTAR score was 8 (IQR = 3). The methodological quality of 47.3% SRs was rated good. SRs without protocol registration (Coef: −3.00, 95% CI: −3.72, −2.28, P < 0.001) and American continent SRs (Coef: −1.00, 95% CI: −1.72, −0.21, P = 0.007) were associated with lower AMSTAR scores. A total of 56.0% SRs were rated a low ROB, with a lower ROB apparent in multicentre SRs (OR: 0.27, 95% CI: 0.11, 0.64, P = 0.003) and a higher ROB evident in SRs without a registered protocol (OR: 111.81, 95% CI: 22.34, 559.62, P < 0.001). When adjusted for the effect of AMSTAR score on ROB, a higher ROB was associated with SRs without protocol registration (OR: 32.24, 95% CI: 6.03, 172.44, P ≤ 0.001). As the AMSTAR score (per unit) increased, the odds of having a high ROB rating decreased (OR: 0.31, 95% CI: 0.21, 0.45, P ≤ 0.001). Conclusions As the methodological quality rating of orthodontic SRs increases, a reduction in the ROB is evident.