The Effect of Examinee Motivation on Test Construction Within an IRT Framework

Abstract

The purpose of this study is to examine the effects of a false assumption regarding the motivation of examinees on test construction. Simulated data were generated using two models of item responses (the three-parameter logistic item response model alone and in combination with Wise’s examinee persistence model) and were calibrated using a Bayesian method. For the conditions studied, biased item parameter estimates resulted from responses from poorly motivated examinees. Bias in item parameter estimates resulted in bias in item information estimates and test information estimates for an optimally constructed test. The direction and magnitude of the bias depended on conditions studied. The implications of the results for test development companies, examinees, and users of test results are discussed.