Deciding the Appropriate Sample Size for Clinical Trials: A complex interplay between power, effect size, and cost

Abstract

Background: Sample size is a key factor in planning a clinical trial. Decisions regarding sample size are typically based on ensuring the statistical power of the test of interest. However, this does not always guarantee a precise estimate of the treatment effect. It is important to understand the distinction between these two aspects of a trial. Methods: Although many computational tools exist for calculating sample size, researchers do not always fully grasp the various issues that must be considered before making a final decision. We propose using simulations to assist in this process. By doing so, researchers can explore different scenarios and better understand the distinction between statistical power and precision in estimating treatment effects. Results: We developed two user-friendly applications using the Shiny package in R. To achieve our goals, we focused on two basic designs: (i) two-arm clinical trials with a binary outcome and (ii) multi-arm clinical trials with a normally distributed outcome. These applications facilitate understanding the selection of sample size and highlight the practical limitations of making decisions based solely on statistical power. Conclusion: Simulation is a useful tool for complementing sample size computation and understanding the possible results associated with that decision. While statistical power is an important concept, decisions on sample size should also consider the precision in estimating treatment effects.