Consequences of Assumption Violations Revisited: A Quantitative Review of Alternatives to the One-Way Analysis of Variance F Test

Abstract

The presence of variance heterogeneity and nonnormality in educational and psychological data may frequently invalidate the use of the analysis of variance (ANOVA) F test in one-way independent groups designs. This article offers recommendations to applied researchers on the use of various parametric and nonparametric alternatives to the F test under assumption violation conditions. Meta-analytic techniques were used to summarize the statistical robustness literature on the Type I error properties of the Brown-Forsythe (Brown & Forsythe, 1974 ), James (1951) second-order, Kruskal-Wallis ( Kruskal & Wallis, 1952 ), and Welch (1951) tests. Two variables, based on the theoretical work of Box (1954) , are shown to be highly effective in deciding when a particular alternative procedure should be adopted. Based on the meta-analysis findings, it is recommended that researchers gain a clear understanding of the nature of their data before conducting statistical analyses. Of all of the procedures, the James and Welch tests performed best under violations of the variance homogeneity assumption, although their sensitivity to certain types of nonnormality may preclude their use in all data-analytic situations. Opportunities for further methodological studies of ANOVA alternative procedures are also discussed.