Comparing an experimental agent to a standard agent: relative merits of a one-arm or randomized two-arm Phase II design

Abstract

Background Phase II clinical trials in cancer are used to assess whether a new agent has sufficiently promising efficacy to proceed on to a larger definitive study comparing the new agent to a standard agent. Purpose A crucial issue in determining the usefulness of a one-arm design is the uncertainty of the historical response rate of the standard therapy. Therefore, we contrast the usual one-arm design of a Phase II trial with a randomized two-arm design that uses the same number of patients. Methods We use simulations and analytical approximations to compare the two designs under a range of realistic values for the historical rate uncertainty and a range of treatment effects. We also extend the simulation model to compare the efficiency of the two designs in settings where multiple Phase II studies are used to make decisions about moving on to a Phase III study. Results For a one-arm design the probability of correctly identifying an effective experimental agent tends to be at least 0.7 in the cases considered, with the corresponding value for a randomized two-arm design within 0.05-0.10 above or below the one-arm design. An increase in total sample size from 30 patients to 80 patients tends to increase the probability of correctly identifying an effective experiment agent more in the two-arm design than the the one-arm design, particularly when the uncertainty in the historical response rate is large. Limitations These results for binary response measures are derived from the specific scenarios and assumptions considered in the simulation study and may not apply to situations outside the range considered. Conclusions We find that a one-arm design is preferred for small sample sizes, but a two-arm design may be preferred with larger sample sizes or if the uncertainty in the historical response rates is large.