A Machine Learning Approach to Assess Differential Item Functioning in Psychometric Questionnaires Using the Elastic Net Regularized Ordinal Logistic Regression in Small Sample Size Groups

Abstract

Assessing differential item functioning (DIF) using the ordinal logistic regression (OLR) model highly depends on the asymptotic sampling distribution of the maximum likelihood (ML) estimators. The ML estimation method, which is often used to estimate the parameters of the OLR model for DIF detection, may be substantially biased with small samples. This study is aimed at proposing a new application of the elastic net regularized OLR model, as a special type of machine learning method, for assessing DIF between two groups with small samples. Accordingly, a simulation study was conducted to compare the powers and type I error rates of the regularized and nonregularized OLR models in detecting DIF under various conditions including moderate and severe magnitudes of DIF ( $\text{DIF}=0.4\text{and}0.8$ ), sample size ( $N$ ), sample size ratio ( $R$ ), scale length ( $I$ ), and weighting parameter ( $w$ ). The simulation results revealed that for $I=5$ and regardless of $R$ , the elastic net regularized OLR model with $w=0.1$ , as compared with the nonregularized OLR model, increased the power of detecting moderate uniform DIF ( $\text{DIF}=0.4$ ) approximately 35% and 21% for $N=100\text{and}150$ , respectively. Moreover, for $I=10$ and severe uniform DIF ( $\text{DIF}=0.8$ ), the average power of the elastic net regularized OLR model with $0.03\le w\le 0.06$ , as compared with the nonregularized OLR model, increased approximately 29.3% and 11.2% for $N=100\text{and}150$ , respectively. In these cases, the type I error rates of the regularized and nonregularized OLR models were below or close to the nominal level of 0.05. In general, this simulation study showed that the elastic net regularized OLR model outperformed the nonregularized OLR model especially in extremely small sample size groups. Furthermore, the present research provided a guideline and some recommendations for researchers who conduct DIF studies with small sample sizes.