Identification of Differential Item Functioning in Assessment Booklet Designs With Structurally Missing Data

Abstract

The Mantel—Haenszel procedure is a popular technique for determining items that may exhibit differential item functioning (DIF). Numerous studies have focused on the strengths and weaknesses of this procedure, but few have focused the performance of the Mantel—Haenszel method when structurally missing data are present as a result of test booklet design. The current research compares the Type I error rate, power rate, and accuracy of DIF statistics when using the Mantel—Haenszel methods to detect DIF in three different test booklet designs that produce sparse data. Data were simulated for three different total sample sizes, fully crossed with three different test lengths, and two focal to reference group ratios. In general, the Mantel—Haenszel procedure was not adversely affected by sparseness of data under any of the booklet designs considered when sample size was adequately large. As sample sizes decrease, both the amount of sparseness and features of the booklet design may influence the ability to detect DIF.