Sequential Anovas and Type I Error Rates

Abstract

Monte Carlo simulations were used to examine three different cases of using analysis of variance sequentially. The first case dealt with adding additional data to original data that were not significant and then reanalyzing. Alpha levels reached a high of .086 in this case. In the second case, data were added only when the F-ratio was not significant but exceeded 1.00; in this case, the alpha level reached a high of .16. The third case added data only when the p value exceeded 0.05 but was less than a predetermined limit (between .06 and .10); alpha levels rose to as much as .39 in this case. The message is clear: ANOVA was not intended as a sequential test and should not be used as such.