An Evidenced-Based Prior for Estimating the Treatment Effect of Phase III Randomized Trials in Oncology

Abstract

ABSTRACTPurposeThe primary results of phase III oncology trials may be challenging to interpret, given that such results are generally based on meetingP-value thresholds. The probability of whether a treatment is beneficial, although a more intuitive summary of the results, is not provided by most trials. In this study, we developed and released a user-friendly tool that calculates the probability that a treatment studied in a phase III oncology trial is beneficial using published summary statistics.MethodsWe curated the primary time-to-event outcomes of 415 phase III, superiority design, therapeutic randomized controlled trials of oncologic treatments enrolling 338,600 patients and published between 2004 and 2020. A phase III oncology-specific prior probability distribution for the treatment effect was developed based on an estimated three-component zero-mean mixture distribution of the observed z-scores. Using this prior, we computed the probability of any benefit (hazard ratio < 1) and the probability of clinically meaningful benefit (hazard ratio < 0.8) for each trial. The distribution of signal-to-noise ratios of phase III oncology trials was compared with that of 23,551 randomized trials from the Cochrane Database of Systematic Reviews.ResultsThe signal-to-noise ratios of phase III oncology trials tended to be much larger than randomized trials from the Cochrane database. Still, the median power of phase III oncology trials was only 49% (IQR, 14% to 95%), and the power was less than 80% in 65% of trials. Using the developed phase III, oncology-specific prior, only 53% of trials claiming superiority (114 of 216) had a ≥ 90% probability of providing clinically meaningful benefits. Conversely, the probability that the experimental arm was superior to the control arm (HR < 1) exceeded 90% in 17% of trials interpreted as having no benefit (34 of 199).ConclusionBy enabling computation of contextual probabilities for the treatment effect from summary statistics, our robust, highly practical tool, now posted on a user-friendly webpage, can aid the wider oncology community in the interpretation of phase III trials.