A robust effect size measure Aw for MANOVA with non-normal and non-homogenous data

Abstract

A common research question in psychology entails examining whether significant group differences (e.g. male and female) can be found in a list of numeric variables that measure the same underlying construct (e.g. intelligence). Researchers often use a multivariate analysis of variance (MANOVA), which is based on conventional null-hypothesis significance testing (NHST). Recently, a number of quantitative researchers have suggested reporting an effect size measure (ES) in this research scenario because of the perceived shortcomings of NHST. Thus, a number of MANOVA ESs have been proposed (e.g. generalized eta squared [Formula: see text], generalized omega squared [Formula: see text]), but they rely on two key assumptions—multivariate normality and homogeneity of covariance matrices—which are frequently violated in psychological research. To solve this problem we propose a non-parametric (or assumptions-free) ES ( Aw) for MANOVA. The new ES is developed on the basis of the non-parametric A in ANOVA. To test Aw we conducted a Monte-Carlo simulation. The results showed that Aw was accurate (robust) across different manipulated conditions—including non-normal distributions, unequal covariance matrices between groups, total sample sizes, sample size ratios, true ES values, and numbers of dependent variables—thereby providing empirical evidence supporting the use of Aw, particularly when key assumptions are violated. Implications of the proposed Aw for psychological research and other disciplines are also discussed.