Checking Dimensionality in Item Response Models With Principal Component Analysis on Standardized Residuals

Abstract

Dimensionality is an important assumption in item response theory (IRT). Principal component analysis on standardized residuals has been used to check dimensionality, especially under the family of Rasch models. It has been suggested that an eigenvalue greater than 1.5 for the first eigenvalue signifies a violation of unidimensionality when there are 500 persons and 30 items. The cut-point of 1.5 is often used beyond this specific condition of sample size and test length. This study argues that a fixed cut-point is not applicable because the distribution of eigenvalues or their ratios depends on sample size and test length, just like other statistics. The authors conducted a series of simulations to verify this argument. They then proposed three chi-square statistics for multivariate independence to test the correlation matrix obtained from the standardized residuals. Through simulations, it was found that Steiger’s statistic behaved fairly like a chi-square distribution, when its degrees of freedom were adjusted.