Testing the Difference Between Two Alpha Coefficients With Small Samples of Subjects and Raters

Abstract

Researchers sometimes need a statistical test of the hypothesis that two values of Cronbach's alpha reliability coefficient are equal. The situation may involve scores from two different measures administered to independent random samples or from the same measure administered to random samples from two different populations. Feldt derived a test that functions well with large or moderate numbers of subjects. However, he validated this test only when the number of parts ( k) of the measurement was fairly large, as it would be if the parts were individual test items. He did not consider instances in which the parts were raters, and hence k would be as small as 2 or 3. In this article, the Feldt test is investigated for such situations. It is found to function quite well in its control of Type I error.