Using IRT Trait Estimates Versus Summated Scores in Predicting Outcomes

Abstract

It has been argued that item response theory trait estimates should be used in analyses rather than number right (NR) or summated scale (SS) scores. Thissen and Orlando postulated that IRT scaling tends to produce trait estimates that are linearly related to the underlying trait being measured. Therefore, IRT trait estimates can be more useful than summated scores when examining relationships between test scores and external variables. Also, when the model holds, IRT trait estimates possess an interval scale that is a property assumed for dependent variables by most statistical procedures used in educational research. The objective of this study was to use Monte Carlo methods to compare the performance of IRT trait estimates and SS scores in predicting outcome variables in the context of health and behavioral assessment. The use of scores based on the graded-response model versus summated scores was compared. Results indicated that IRT-based scores and summated scores are comparable when evaluating the relationships between test scores and outcome measures. Thus, applied researchers could use summated scores in predictive studies and circumvent evaluating the assumptions underlying use of IRT-based scores.