Multidimensional item Response Theory Calibration of Dichotomous Response Structure Using R Language for Statistical Computing

Abstract

AbstractThe study focuses on assessing the proficiency levels of higher education students, specifically the physics achievement test (PHY 101) at the National Open University of Nigeria (NOUN). This test, like others, evaluates various aspects of knowledge and skills simultaneously. However, relying on traditional models for such tests can result in inaccurate interpretations of students’ abilities. The research highlights the importance of exploring the multidimensional nature of the PHY 101 test to improve its accuracy in measuring student proficiency and enhance education and assessment quality at NOUN. Using an ex-post facto research design, the study analyzed 978 responses from NOUN’s Directorate of Examination and Assessment. Through confirmatory and exploratory DETECT techniques, the study found strong evidence supporting the test’s multidimensionality. Three distinct dimensions emerged: cognitive processing, reading ability, and problem-solving skills. A parsimonious multidimensional three-parameter logistic model was used to calibrate the test items, providing valuable insights into item difficulty, discrimination, and resistance to chance influences. While the study primarily focuses on the psychometric aspects of the PHY 101 test, it is important to consider its broader impact on the educational community. The research contributes to educational assessment by emphasizing the significance of recognizing and addressing the multidimensional nature of higher education tests. This approach can result in more accurate assessments of students’ abilities, ultimately improving education quality and fairness. The findings confirm the multidimensional nature of the PHY 101 test and identify three distinct dimensions, aligning with the study’s objective. These insights are relevant to educators and test developers, highlighting the need for a multidimensional approach to effectively assess and enhance student proficiency. For researchers interested in similar studies, it is recommended to explore the broader influence of multidimensional models in educational assessment. Investigating their impact on teaching methods, curriculum development, and student learning experiences can provide valuable insights. Longitudinal studies assessing the long-term effects of multidimensional assessment on student outcomes and success are also recommended.