Assessing the Dimensionality of a Set of Test Items

Abstract

This study compared four methods of determining the dimensionality of a set of test items: linear factor analysis, nonlinear factor analysis, residual analysis, and a method developed by Bejar (1980). Five artifi cial test datasets (for 40 items and 1,500 examinees) were generated to be consistent with the three-parame ter logistic model and the assumption of either a one- or a two-dimensional latent space. Two variables were manipulated: (1) the correlation between the traits (r = .10 or r = .60) and (2) the percent of test items measuring each trait (50% measuring each trait, or 75% measuring the first trait and 25% measuring the second trait). While linear factor analysis in all instances over estimated the number of underlying dimensions in the data, nonlinear factor analysis with linear and quad ratic terms led to correct determination of the item di mensionality in the three datasets where it was used. Both the residual analysis method and Bejar's method proved disappointing. These results suggest the need for extreme caution in using linear factor analysis, re sidual analysis, and Bejar's method until more investi gations of these methods can confirm their adequacy. Nonlinear factor analysis appears to be the most prom ising of the four methods, but more experience in ap plying the method seems necessary before wide-scale use can be recommended.