Network Meta-analysis on Disconnected Evidence Networks When Only Aggregate Data Are Available: Modified Methods to Include Disconnected Trials and Single-Arm Studies while Minimizing Bias

Abstract

Background Network meta-analysis (NMA) requires a connected network of randomized controlled trials (RCTs) and cannot include single-arm studies. Regulators or academics often have only aggregate data. Two aggregate data methods for analyzing disconnected networks are random effects on baseline and aggregate-level matching (ALM). ALM has been used only for single-arm studies, and both methods may bias effect estimates. Methods We modified random effects on baseline to separate RCTs connected to and disconnected from the reference and any single-arm studies, minimizing the introduction of bias. We term our modified method reference prediction. We similarly modified ALM and extended it to include RCTs disconnected from the reference. We tested these methods using constructed data and a simulation study. Results In simulations, bias for connected treatments for ALM ranged from −0.0158 to 0.051 and for reference prediction from −0.0107 to 0.083. These were low compared with the true mean effect of 0.5. Coverage ranged from 0.92 to 1.00. In disconnected treatments, bias of ALM ranged from −0.16 to 0.392 and of reference prediction from −0.102 to 0.40, whereas coverage of ALM ranged from 0.30 to 0.82 and of reference prediction from 0.64 to 0.94. Under fixed study effects for disconnected evidence, bias was similar, but coverage was 0.81 to 1.00 for reference prediction and 0.18 to 0.76 for ALM. Trends of similar bias but greater coverage for reference prediction with random study effects were repeated in constructed data. Conclusions Both methods with random study effects seem to minimize bias in treatment connected to the reference. They can estimate treatment effects for disconnected treatments but may be biased. Reference prediction has greater coverage and may be recommended overall. Highlights Two methods were modified for network meta-analysis on disconnected networks and for including single-arm observational or interventional studies in network meta-analysis using only aggregate data and for minimizing the bias of effect estimates for treatments only in trials connected to the reference. Reference prediction was developed as a modification of random effects on baseline that keeps analyses of trials connected to the reference separately from those disconnected from the reference and from single-arm studies. The method was further modified to account for correlation in trials with more than 2 arms and, under random study effects, to estimate variance in heterogeneity separately in connected and disconnected evidence. Aggregate-level matching was extended to include trials disconnected from the reference, rather than only single-arm studies. The method was further modified to separately estimate treatment effects and heterogeneity variance in the connected and disconnected evidence and to account for the correlation between arms in trials with more than 2 arms. Performance was assessed using a constructed data example and simulation study. The methods were found to have similar, and sometimes low, bias when estimating the relative effects for disconnected treatments, but reference prediction with random study effects had the greatest coverage. The use of reference prediction with random study effects for disconnected networks is recommended if no individual patient data or alternative real-world evidence is available.