Unidimensional IRT Calibration of Compensatory and Noncompensatory Multidimensional Items

Abstract

The characteristics of unidimensional ability esti mates obtained from data generated using multidimen sional compensatory models were compared with esti mates from noncompensatory IRT models. Reckase, Carlson, Ackerman, and Spray (1986) reported that when a compensatory model is used and item diffi culty is confounded with dimensionality, the composi tion of the unidimensional ability estimates differs for different points along the unidimensional ability (θ) scale. Eight datasets (four compensatory, four non compensatory) were generated for four different levels of correlated two-dimensional θs. In each dataset, dif ficulty was confounded with dimensionality and then calibrated using LOGIST and BILOG. The confounding of difficulty and dimensionality affected the BILOG cal ibration of response vectors using matched multidi mensional item parameters more than it affected the LOGIST calibration. As the correlation between the generated two-dimensional θs increased, the response data became more unidimensional as shown in bivar iate plots of the mean θ1 as opposed to the mean of θ2 for specified unidimensional quantiles. Index terms: BILOG, compensatory IRT models, IRT ability estima tion, LOGIST, multidimensional item response theory, noncompensatory IRT models.