Conditional Covariance-Based Subtest Selection for DIMTEST

Abstract

DIMTEST is a nonparametric hypothesis-testing procedure designed to test the assumptions of a unidimensional and locally independent item response theory model. Several previous Monte Carlo studies have found that using linear factor analysis to select the assessment subtest for DIMTEST results in a moderate to severe loss of power when the exam lacks simple structure, the ability and difficulty parameter distributions differ greatly, or the underlying model is noncompensatory. A new method of selecting the assessment subtest for DIMTEST, based on the conditional covariance dimensionality programs DETECT and HCA/ CCPROX, is presented. Simulation studies show that using DIMTEST with this new selection method has either similar or significantly higher power to detect multidimensionality than using linear factor analysis for subtest selection, while maintaining Type I error rates around the nominal level.