Using real‐world data to predict health outcomes—The prediction design: Application and sample size planning

Abstract

AbstractThe gold standard for investigating the efficacy of a new therapy is a (pragmatic) randomized controlled trial (RCT). This approach is costly, time‐consuming, and not always practicable. At the same time, huge quantities of available patient‐level control condition data in analyzable format of (former) RCTs or real‐world data (RWD) are neglected. Therefore, alternative study designs are desirable. The design presented here consists of setting up a prediction model for determining treatment effects under the control condition for future patients. When a new treatment is intended to be tested against a control treatment, a single‐arm trial for the new therapy is conducted. The treatment effect is then evaluated by comparing the outcomes of the single‐arm trial against the predicted outcomes under the control condition. While there are obvious advantages of this design compared to classical RCTs (increased efficiency, lower cost, alleviating participants’ fear of being on control treatment), there are several sources of bias. Our aim is to investigate whether and how such a design—the prediction design—may be used to provide information on treatment effects by leveraging external data sources. For this purpose, we investigated under what assumptions linear prediction models could be used to predict the counterfactual of patients precisely enough to construct a test and an appropriate sample size formula for evaluating the average treatment effect in the population of a new study. A user‐friendly R Shiny application (available at: https://web.imbi.uni‐heidelberg.de/PredictionDesignR/) facilitates the application of the proposed methods, while a real‐world application example illustrates them.