Bayesian Interim Analysis and Efficiency of Phase III Randomized Trials

Abstract

ABSTRACTIMPORTANCEImproving the efficiency of interim assessments in phase III trials should reduce trial costs, hasten the approval of efficacious therapies, and mitigate patient exposure to disadvantageous randomizations.OBJECTIVEWe hypothesized thatin silicoBayesian early stopping rules improve the efficiency of phase III trials compared with the original frequentist analysis without compromising overall interpretation.DESIGNCross-sectional analysis.SETTING230 randomized phase III oncology trials enrolling 184,752 participants.PARTICIPANTSIndividual patient-level data were manually reconstructed from primary endpoint Kaplan-Meier curves.INTERVENTIONSTrial accruals were simulated 100 times per trial and leveraged published patient outcomes such that only the accrual dynamics, and not the patient outcomes, were randomly varied.MAIN OUTCOMES AND MEASURESEarly stopping was triggered per simulation if interim analysis demonstrated ≥ 85% probability of minimum clinically important difference/3 for efficacy or futility. Trial-level early closure was defined by stopping frequencies ≥ 0.75.RESULTSA total of 12,451 simulations (54%) met early stopping criteria. Trial-level early stopping frequency was highly predictive of the published outcome (OR, 7.24; posterior probability of association, >99.99%; AUC, 0.91;P< 0.0001). Trial-level early closure was recommended for 82 trials (36%), including 62 trials (76%) which had performed frequentist interim analysis. Bayesian early stopping rules were 96% sensitive (95% CI, 91% to 98%) for detecting trials with a primary endpoint difference, and there was a high level of agreement in overall trial interpretation (Bayesian Cohen’s κ, 0.95; 95% CrI, 0.92 to 0.99). However, Bayesian interim analysis was associated with >99.99% posterior probability of reducing patient enrollment requirements (P< 0.0001), with an estimated cumulative enrollment reduction of 20,543 patients (11%; 89 patients averaged equally over all studied trials) and an estimated cumulative cost savings of 851 million USD (3.7 million USD averaged equally over all studied trials).CONCLUSIONS AND RELEVANCEBayesian interim analyses may improve randomized trial efficiency by reducing enrollment requirements without compromising trial interpretation. Increased utilization of Bayesian interim analysis has the potential to reduce costs of late-phase trials, reduce patient exposures to ineffective therapies, and accelerate approvals of effective therapies.KEY POINTSQuestionWhat are the effects of Bayesian early stopping rules on the efficiency of phase III randomized oncology trials?FindingsIndividual-patient level outcomes were reconstructed for 184,752 patients from 230 trials. Compared with the original interim analysis strategy,in silicoBayesian interim analysis reduced patient enrollment requirements and preserved the original trial interpretation.MeaningBayesian interim analysis may improve the efficiency of conducting randomized trials, leading to reduced costs, reduced exposure of patients to disadvantageous treatments, and accelerated approval of efficacious therapies.