Detecting Differential Item Functioning in CAT Using IRT Residual DIF Approach

Abstract

AbstractThe residual differential item functioning (RDIF) detection framework was developed recently under a linear testing context. To explore the potential application of this framework to computerized adaptive testing (CAT), the present study investigated the utility of the RDIFR statistic both as an index for detecting uniform DIF of pretest items in CAT and as a direct measure of the effect size of uniform DIF. Extensive CAT simulations revealed RDIFR to have well‐controlled Type I error and slightly higher power to detect uniform DIF compared with CATSIB, especially when pretest items were calibrated using fixed‐item parameter calibration. Moreover, RDIFR accurately estimated the amount of uniform DIF irrespective of the presence of impact. Therefore, RDIFR demonstrates its potential as a useful tool for evaluating both the statistical and practical significance of uniform DIF in CAT.