A sequential Bayesian changepoint detection procedure for aberrant behaviours in computerized testing

Abstract

AbstractChangepoints are abrupt variations in a sequence of data in statistical inference. In educational and psychological assessments, it is essential to properly differentiate examinees' aberrant behaviours from solution behaviour to ensure test reliability and validity. In this paper, we propose a sequential Bayesian changepoint detection algorithm to monitor the locations of changepoints for response times in real time and, subsequently, further identify types of aberrant behaviours in conjunction with response patterns. Two simulation studies were conducted to investigate the efficiency and accuracy of the proposed detection procedure in terms of identifying one or multiple changepoints at different locations. In addition to manipulating the number and locations of changepoints, two types of aberrant behaviours were also considered: rapid guessing behaviour and cheating behaviour. Simulation results indicate that ability estimates could be improved after removing responses from aberrant behaviours identified by our approach. Two empirical examples were analysed to illustrate the application of the proposed sequential Bayesian changepoint detection procedure.