Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials

Abstract

ABSTRACTMost oncology trials define superiority of an experimental therapy compared to a control therapy according to frequentist significance thresholds, which are widely misinterpreted. Posterior probability distributions computed by Bayesian inference may be more intuitive measures of uncertainty, particularly for measures of clinical benefit such as the minimum clinically important difference (MCID). Here, we manually reconstructed 194,129 individual patient-level outcomes across 230 phase III, superiority-design, oncology trials. Posteriors were calculated by Markov Chain Monte Carlo sampling using standard priors. All trials interpreted as positive had probabilities > 90% for marginal benefits (HR < 1). However, 38% of positive trials had ≤ 90% probabilities of achieving the MCID (HR < 0.8), even under an enthusiastic prior. A subgroup analysis of 82 trials that led to regulatory approval showed 30% had ≤ 90% probability for meeting the MCID under an enthusiastic prior. Conversely, 24% of negative trials had > 90% probability of achieving marginal benefits, even under a skeptical prior, including 12 trials with a primary endpoint of overall survival. Lastly, a phase III oncology-specific prior from a previous work, which uses published summary statistics rather than reconstructed data to compute posteriors, validated the individual patient-level data findings. Taken together, these results suggest that Bayesian models add considerable unique interpretative value to phase III oncology trials and provide a robust solution for overcoming the discrepancies between refuting the null hypothesis and obtaining a MCID.SIGNIFICANCE STATEMENTThe statistical analyses of oncology trials are usually performed by calculatingPvalues, although these are poorly understood. UsingPvalue cutoffs, such asP< 0.05, may lead to some treatments being accepted which have little benefit, and other therapies being rejected which have considerable benefit. A more intuitive and direct probability— that an experimental treatment is better than a standard treatment—can be calculated by Bayesian statistics. Here we used software to obtain the outcomes of 194,129 patients enrolled across 230 trials and then calculated probabilities of benefit. Interpretations based onPvalues disagreed with the probabilities of benefit in one-third of trials. This study suggests that probabilities of benefit would considerably enhance the interpretation of oncology trials.