Testing the Equality of Independent Alpha Coefficients Adjusted for Test Length

Abstract

In comparing the merits of different item types or test formats, test developers are often called on to make a rigorous comparison of the internal consistency reliabilities of the scores from two competing tests. Such comparisons should be made for tests of equal length in terms of time demands and speededness. If the experimental instruments used in the field trials were not equal in length, estimates of the reliabilities of scores on equal-length tests can be obtained by applying the Spearman-Brown formula. In this article, the authors develop a statistical test for the hypothesis that [.alpha]1[.minute] = [.alpha]2[.minute], where [.alpha]i[.minute] is the extrapolated value of Cronbach’s alpha reliability for Test i. The test is shown to exercise tight control of Type I error.