Detecting Heterogeneity in Logistic Regression Models

Abstract

In the context of item response theory, it is not uncommon that person-by-item data are correlated beyond the correlation that is captured by the model—in other words, there is extra binomial variation. Heterogeneity of the parameters can explain this variation. There is a need for proper statistical methods to indicate possible extra heterogeneity and its location because investigating all different combinations of random parameters is not practical and sometimes even unfeasible. The ignored random person effects are the focus of this study. Considering the random weights linear logistic test model, random effects can occur as a general latent trait and as weights of covariates. A simulation study was conducted with different sources and degrees of heterogeneity to investigate and compare various methods: individual analyses (one per person), marginal modeling, principal component analysis of the raw data, DIMTEST, and DETECT. The methods are illustrated with an application on deductive reasoning.