The importance of test order in external and standardized test results: The case of PISA 2018

Abstract

Standardized tests intend to reduce information asymmetry by providing a common and objective measure of students’ academic performance. The basic assumption underlying standardized testing is that differences in student performance on standardized tests should be attributed primarily to differences in the quality of education received by students. However, there is evidence that environmental factors can affect standardized test scores, which may result in anomalous observations or outliers that show a distortion of student performance. In this regard, the exclusion of Spain from PISA 2018 is particularly interesting as Spanish data met PISA 2018 Technical Standards but showed implausible student-response behavior. The aim of this paper is to complement the OECD’s analysis of Spain’s exclusion from PISA 2018 by exploring the potential reasons behind the outlier results, focusing on the Madrid region.