A Comparative Study of Test Data Dimensionality Assessment Procedures Under Nonparametric IRT Models

Abstract

In this article, an overview of nonparametric item response theory methods for determining the dimensionality of item response data is provided. Four methods were considered: MSP, DETECT, HCA/CCPROX, and DIMTEST. First, the methods were compared theoretically. Second, a simulation study was done to compare the effectiveness of MSP, DETECT, and HCA/CCPROX using the default settings of each program in finding a simulated dimensional structure of a matrix of item response data. In several design cells, the methods that use covariances conditional on the latent trait (DETECT and HCA/CCPROX) were superior in finding the simulated structure to the method that used normed unconditional covariances (MSP). Third, the correctness of the decision of accepting or rejecting unidimensionality based on the statistics used in DETECT and DIMTEST was considered. This decision did not always reflect the true dimensionality of the item pool. Index terms: DETECT software and method, dimensionality of item response data, DIMTEST software and method, HCA/CCPROX software and method, MSP software and method, multidimensional item response data, nonparametric item response theory, unidimensional item response data.