Significance Testing of Subtest Score Differences: The Case of Nonsignificant Results

Abstract

Though the testing for the statistical significance of subtest score differences yields a considerable proportion of nonsignificant results (e.g., about two-thirds of the V-P IQ differences on the WISC-R, assuming α = .05), the interpretation of such results has been neglected. This paper examines the implications of the decision rule as they concern statistically nonsignificant results. The acceptance of the null hypothesis after a nonsignificant difference is shown to result in a noticeable proportion of both Type II and classification errors. Thus, in view of its low statistical power and the considerable proportion of classification errors involved, the widespread use of significance testing of subtest score differences is questioned; it cannot be justified in terms of a very extreme loss function, which emphasizes the minimization of the α-probability of Type I errors at the expense of the β-probability of Type II errors. Two other explanations for the universal acceptance of the significance testing of subtest score differences are suggested and critically examined.