Increasing the Degrees of Freedom in Future Group Randomized Trials

Abstract

Background: This article revisits an article published in Evaluation Review in 2005 on sample size estimation and power analysis for group-randomized trials. With help from a careful reader, we learned of an important error in the spreadsheet used to perform the calculations and generate the results presented in that article. As we studied the spreadsheet, we discovered other minor errors. When we corrected the errors, we found that the results were substantially different and that the conclusions reported in the original article were not always appropriate. Objective: This article corrects the errors and reports the results as they should have been reported originally. Method: Using a random-effects meta-analytic model, estimates of intraclass correlation were combined from two studies to guide sample size calculations for a new study. Results: The df* method can result in improved power or smaller studies when used a priori to plan future group-randomized trials, though the improvements will be modest in larger studies and will likely be insufficient to provide adequate power to small studies. Conclusion: Smaller group-randomized trials are often desirable, for example, as pilot studies to help plan for a full-scale efficacy trial, as replication studies, or in situations in which resource constraints prohibit a larger trial. We discuss the circumstances under which the df* method will be most helpful and the risks associated with conducting smaller studies.