Modeling Response Styles in Cross‐Classified Data Using a Cross‐Classified Multidimensional Nominal Response Model

Abstract

AbstractIn this study, we introduced a cross‐classified multidimensional nominal response model (CC‐MNRM) to account for various response styles (RS) in the presence of cross‐classified data. The proposed model allows slopes to vary across items and can explore impacts of observed covariates on latent constructs. We applied a recently developed variant of the Metropolis‐Hastings Robbins‐Monro (MH‐RM) algorithm to address the computational challenge of estimating the proposed model. To demonstrate our new approach, we analyzed empirical student evaluation of teaching (SET) data collected from a large public university with three models: a CC‐MNRM with RS, a CC‐MNRM with no RS, and a multilevel MNRM with RS. Results indicated that the three models led to different inferences regarding the observed covariates. Additionally, in the example, ignoring/incorporating RS led to changes in student substantive scores, while the instructor substantive scores were less impacted. Misspecifying the cross‐classified data structure resulted in apparent changes on instructor scores. To further evaluate the proposed modeling approach, we conducted a preliminary simulation study and observed good parameter and score recovery. We concluded this study with discussions of limitations and future research directions.