Evaluating measurement invariance of students’ practices regarding online information questionnaire in PISA 2022: a comparative study using MGCFA and alignment method

Abstract

AbstractInternational large-scale assessments provide cross-national data on students’ cognitive and non-cognitive characteristics. A critical methodological issue that often arises in comparing data from cross-national studies is ensuring measurement invariance, indicating that the construct under investigation is the same across the compared groups. This study addresses the measurement invariance of students’ practices regarding online information (ICTINFO) questionnaire across countries in the PISA 2022 cycle. Some methodological complexities have arisen when testing the measurement invariance across the presence of many groups. For testing measurement invariance, the multiple group confirmatory factor analysis (MGCFA), which is a traditional procedure, was employed first, and then a novel approach, the alignment method, was performed. This study comprised 29 OECD countries, with a total sample size of 187.614 15-year-old students. The MGCFA results revealed that metric invariance was achieved across countries, indicating comparable factor loadings while not the same for factor means. Consistent with MGCFA results, the alignment method identified noninvariant parameters exceeding the 25% cut-off criteria across countries. Monte Carlo simulation validated the reliability of the alignment results. This study contributes to international assessments by providing a detailed examination of measurement invariance and comparing the findings from various methodologies for improving assessment accuracy. The results provide evidence-based recommendations for policymakers to ensure fair and equitable evaluations of student performance across different countries, thereby contributing to more reliable and valid international assessments.