Power Analysis for Ordinal Analyses of the Modified Rankin Scale and an Online and Downloadable Tool for Practical Use

Abstract

BACKGROUND: Several methods for conducting power analysis of studies with outcomes across the full ordinal modified Rankin Scale are proposed in the literature. No systematic comparison of accuracy, agreement, and sensitivity to small changes in hypothesized effect sizes for these methods is available. Our aim is to conduct such a systematic comparative analysis and to introduce a comprehensive freely available online tool to facilitate appropriate power analyses for ordinal outcomes. METHODS: We performed simulation studies utilizing the control arm modified Rankin Scale distributions from the AVERT (A Very Early Rehabilitation Trial), EXTEND (Extending the Time for Thrombolysis in Emergency Neurological Deficits), and HERMES (Highly Effective Reperfusion Evaluated in Multiple Endovascular Stroke Trials) studies, as well as a uniform distribution, in combination with hypothetical treatment effects. We systematically evaluated published power formulas for Ordinal Logistic Regression and Tournament Methods (generalized odds ratio; Win Probability; Win Ratio; and Wilcoxon-Mann-Whitney U test). We also developed an online and downloadable Shiny R app facilitating sample size calculation for, and ordinal analysis of, modified Rankin Scale data. RESULTS: Power formulas for Tournament Methods performed well, while the formula for ordinal logistic regression was inaccurate. Tang’s Wilcoxon-Mann-Whitney U test sample size formula exhibited the highest accuracy. All methods, including ordinal logistic regression, had almost identical empirical power for a given sample size. All power methods exhibited sensitivity to small changes in hypothesized effect size. The developed freely available online app supports analytical and visualization requirements for all investigated methods for power and statistical analyses of ordinal modified Rankin Scale outcomes. CONCLUSIONS: As Tournament Method sample size formulas are assumption-free and accurately calculate power, stroke researchers should use these methods when designing studies with outcomes measured on the full or partially collapsed modified Rankin Scale as well as other ordinal scales, even if they intend to use ordinal logistic regression for analysis. Conducting sensitivity analyses of the effect size assumptions are essential for appropriate sample size estimation. Our developed tool supports both of these recommendations ( https://www.thembc.com.au/tournamentmethods ).