Summarizing Monte Carlo Results in Methodological Research: The Single-Factor, Fixed-Effects ANCOVA Case

Abstract

Results from exact statistical theory and Monte Carlo studies have provided evidence that the test size and power of the F test in analysis of covariance are sensitive to violations of certain assumptions. However, a comprehensive summary of the effect of assumption violations has not been available. In this article, meta-analytic methods are used to summarize the results of Monte Carlo studies of the test size and power of the F test in the single-factor, fixed-effects analysis of covariance model, updating and extending narrative reviews of this literature. Monte Carlo results for the nonparametric rank-transform test in the analysis of covariance model are also analyzed. Guidelines for using these tests when assumptions are violated are presented to promote more judicious use of these procedures.