Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study

Abstract

Abstract Background Network meta-analyses can be valuable for decision-makers in guiding clinical practice. However, for network meta-analysis results to be reliable, the assumptions of both transitivity and coherence must be met, and the methodology should adhere to current best practices. We aimed to assess whether network meta-analyses of randomized controlled trials (RCTs) comparing interventions for proximal humerus fractures provide reliable estimates of intervention effects. Methods We searched PubMed, EMBASE, The Cochrane Library, and Web of Science for network meta-analyses comparing interventions for proximal humerus fractures. We critically assessed the methodology regarding the development of a protocol, search strategy, trial inclusion, outcome extraction, and the methods used to conduct the network meta-analyses. We assessed the transitivity and coherence of the network graphs for the Constant score (CS), Disabilities of the Arm, Shoulder, and Hand score (DASH), and additional surgery. Transitivity was assessed by comparing probable effect modifiers (age, gender, fracture morphology, and comorbidities) across intervention comparisons. Coherence was assessed using Separating Indirect from Direct Evidence (SIDE) (Separating Indirect from Direct Evidence) and the design-by-treatment interaction test. We used CINeMA (Confidence in Network Meta-analyses) to assess the confidence in the results. Results None of the three included network meta-analyses had a publicly available protocol or data-analysis plan, and they all had methodological flaws that could threaten the validity of their results. Although we did not detect incoherence for most comparisons, the transitivity assumption was violated for CS, DASH, and additional surgery in all three network meta-analyses. Additionally, the confidence in the results was ‘very low’ primarily due to within-study bias, reporting bias, intransitivity, imprecision, and heterogeneity. Conclusions Current network meta-analyses of RCTs comparing interventions for proximal humerus fractures do not provide reliable estimates of intervention effects. We advise caution in using these network meta-analyses to guide clinical practice. To improve the utility of network meta-analyses to guide clinical practice, journal editors should require that network meta-analyses are done according to a predefined analysis plan in a publicly available protocol and that both coherence and transitivity have been adequately assessed and reported.