Type I Error Rate and Power of Some Alternative Methods to the Independent Samples t Test

Abstract

Several tests exist for use in treatment-control studies in which a larger treatment mean may be accompanied by a larger treatment variance. Type I error rates and power were estimated for four of these tests and for the independent samples t test and the Welch test. Five factors were manipulated: heteroscedasticity, total sample size, sample size imbalance, distribution, and power. The test due to Brownie, Boos, and Hughes-Oliver may not control the Type I error rate when the data are skewed. Among the other tests, one due to Conover and Salsburg and one due to Johnson, Verrill, and Moore had the best power advantage when compared to the independent sample t test and the Welch test. In the Johnson et al. test, a parameter that strongly affects power is specified, and a poor choice can reduce power below that for the Conover and Salsburg test.