Pilot Trial Characteristics, Postpilot Design Modifications, and Feasibility of Full-Scale Trials

Abstract

ImportancePilot trials often lead to study design changes in subsequent full-scale trials. Yet, it remains unclear whether these modifications improve the feasibility of the larger trial.ObjectiveTo compare feasibility estimates between pilot and full-scale trials and identify pilot trial characteristics and modifications associated with equivalent or improved feasibility in the full-scale trial.Design, Setting, and ParticipantsThis cohort study used pilot trials published between January 2005 and December 2018 and their corresponding full-scale trials. PubMed was searched for trials on February 19, 2022.ExposuresPilot trial characteristics and postpilot trial design modifications.Main Outcomes and MeasuresThe outcome of interest was difference in 3 feasibility parameters: successful screening probability, enrollment rate, and retention probability. These metrics were defined as equivalent or improved if the full-scale trial’s estimate was within or exceeding 10% of the pilot trial’s estimate.ResultsA total of 249 pairs of pilot and full-scale trials were analyzed, with 43%, 77%, and 82% of full-scale trials having equivalent or improved successful screening probabilities, enrollment rates, and retention probabilities, respectively. When pilot trials used feasibility progression criteria (relative risk [RR], 1.94; 95% CI, 1.02-5.97) and maintained masking for participants (RR, 1.82; 95% CI, 1.04-4.33) or health care practitioners (RR, 1.81; 95% CI, 1.03-3.97) consistent with the full-scale trial, the likelihood of achieving equivalent or improved screening success in full-scale trials increased. Increasing study sites after the pilot was associated with higher likelihood of equivalent or improved enrollment rates (RR, 1.03; 95% CI, 1.01-1.08). Adding extra content to the intervention (RR, 0.82; 95% CI, 0.66-0.98), changing to active control (RR, 0.74; 95% CI, 0.48-0.99), administrating the control treatment more frequently (RR, 0.60; 95% CI, 0.29-0.93), different follow-up lengths (RR, 0.85; 95% CI, 0.73-0.97), and more follow-up visits (RR, 0.86; 95% CI, 0.75-0.98) were associated with lower likelihood of equivalent or improved retention probability in the full-scale trial.Conclusions and relevanceIn this cohort study of pilot and full-scale trial pairs, pilot trial characteristics and postpilot modifications had varying associations with full-scale trial’s feasibility. If full-scale trials planned for masking, it was desirable to use it in the pilot. Modifications increasing participant burden might decrease full-scale trial feasibility. Trialists and funders should consider both pilot trial data and proposed design changes when assessing full-scale trials.