Affiliation:
1. School of Mathematics and Statistics Southwest University Chongqing China
2. Basic Education Research Centre Southwest University Chongqing China
3. Collaborative Innovation Centre of Assessment for Basic Education Quality Southwest University Branch Chongqing China
4. Collaborative Innovation Centre of Assessment for Basic Education Quality Beijing Normal University Beijing China
Abstract
AbstractQ‐matrix is an important component of most cognitive diagnosis models (CDMs); however, it mainly relies on subject matter experts' judgements in empirical studies, which introduces the possibility of misspecified q‐entries. To address this, statistical Q‐matrix validation methods have been proposed to aid experts' judgement. A few of these methods, including the multiple logistic regression‐based (MLR‐B) method and the Hull method, can be applied to general CDMs, but they are either time‐consuming or lack accuracy under certain conditions. In this study, we combine the L1 regularization and MLR model to validate the Q‐matrix. Specifically, an L1 penalty term is imposed on the log‐likelihood of the MLR model to select the necessary attributes for each item. A simulation study with various factors was conducted to examine the performance of the new method against the two existing methods. The results show that the regularized MLR‐B method (a) produces the highest Q‐matrix recovery rate (QRR) and true positive rate (TPR) for most conditions, especially with a small sample size; (b) yields a slightly higher true negative rate (TNR) than either the MLR‐B or the Hull method for most conditions; and (c) requires less computation time than the MLR‐B method and similar computation time as the Hull method. A real data set is analysed for illustration purposes.
Funder
Humanities and Social Science Fund of Ministry of Education of China
National Natural Science Foundation of China