Hypothetical Use of Multidimensional Adaptive Testing for the Assessment of Student Achievement in the Programme for International Student Assessment

Abstract

The usefulness of multidimensional adaptive testing (MAT) for the assessment of student literacy in the Programme for International Student Assessment (PISA) was examined within a real data simulation study. The responses of N = 14,624 students who participated in the PISA assessments of the years 2000, 2003, and 2006 in Germany were used to simulate MAT with different restrictions (unrestricted, treatment of link items, treatment of open items, content balance, unitwise item selection, all restrictions). Compared with conventional testing based on the booklet design of PISA 2006, unrestricted MAT increases measurement efficiency by 74% and reduces the average number of presented items from 55 to 26 without a loss in measurement precision. The incorporation of restrictions reduces the advantages of MAT. MAT is recommended for the assessment of newly introduced constructs but not for the assessment of the literacy domains in PISA.