Effects of sceptical priors on the performance of adaptive clinical trials with binary outcomes.

Abstract

AbstractIt is unclear how sceptical priors impact adaptive trials. We assessed the influence of priors expressing a spectrum of scepticism on the performance of several Bayesian, multi‐stage, adaptive clinical trial designs using binary outcomes under different clinical scenarios. Simulations were conducted using fixed stopping rules and stopping rules calibrated to keep type 1 error rates at approximately 5%. We assessed total sample sizes, event rates, event counts, probabilities of conclusiveness and selecting the best arm, root mean squared errors (RMSEs) of the estimated treatment effect in the selected arms, and ideal design percentages (IDPs; which combines arm selection probabilities, power, and consequences of selecting inferior arms), with RMSEs and IDPs estimated in conclusive trials only and after selecting the control arm in inconclusive trials. Using fixed stopping rules, increasingly sceptical priors led to larger sample sizes, more events, higher IDPs in simulations ending in superiority, and lower RMSEs, lower probabilities of conclusiveness/selecting the best arm, and lower IDPs when selecting controls in inconclusive simulations. With calibrated stopping rules, the effects of increased scepticism on sample sizes and event counts were attenuated, and increased scepticism increased the probabilities of conclusiveness/selecting the best arm and IDPs when selecting controls in inconclusive simulations without substantially increasing sample sizes. Results from trial designs with gentle adaptation and non‐informative priors resembled those from designs with more aggressive adaptation using weakly‐to‐moderately sceptical priors. In conclusion, the use of somewhat sceptical priors in adaptive trial designs with binary outcomes seems reasonable when considering multiple performance metrics simultaneously.