A Comparison of Item-Fit Statistics for the Three-Parameter Logistic Model

Abstract

In this article, the Type I error rate and the power of a number of existing and new tests of fit to the 3-parameter logistic model (3PLM) are investigated. The first test is a generalization of a test for the evaluation of the fit to the 2-parameter logistic model (2PLM) based on the Lagrange multiplier (LM) test or the equivalent efficient score test. This technique is applied to two model violations: deviation from the 3PLM item characteristic curve and violation of local stochastic independence. The LM test for the first violation is compared with the Q1 – G² j and S – G² j tests, respectively. The LM test for the second violation is compared with the Q3 test and a new test, the S3 test, which can be viewed as a generalization of the approach of the S – G² j test to the evaluation of violation of local independence. The results of simulation studies indicate that all tests, except the Q1 – G² j test, have a Type I error rate that is acceptably close to the nominal significance level, and good power to detect the model violations they are targeted at. When, however, misfitting items are present in a test, the proportion of items that are flagged incorrectly as misfitting can become undesirably high, especially for short tests.