Alternative Hypothesis Testing Procedures for DIMTEST

Abstract

Many commonly used item response models make the unidimensionality assumption of a single latent trait underlying the response data. The validity of this assumption needs to be tested before these models can be applied. One option is to use Stout’s non-parametric hypothesis test of essential unidimensionality, which is operationalized in the DIMTEST procedure. Although generally successful, Type I error rates of this procedure are usually deflated in small samples and inflated in large samples, while power can be low in small samples. A possible cause for the unfavorable Type I error rates and power may be that estimates of the sampling distribution, bias, and standard error of the test statistic are not sufficiently accurate in finite samples. In this study, five alternative hypothesis testing procedures were formulated that replace the (asymptotically correct) approximations in the current DIMTEST procedure with computational alternatives. The performance of these procedures was investigated in two simulation studies. One of these alternative procedures, which uses a conditional covariance statistic directly in a bootstrap hypothesis test, exhibited better controlled Type I errors and higher power than the current DIMTEST procedure in most conditions. Averaged over all sample sizes and correlations between two underlying dimensions, power increased by 5 percentage points for simple structure and by 7 percentage points for approximate simple structure.