Rasch Analysis for Standards-Setting Appraisal of Competency Level-Based Performance on the Part of Instructors in Higher Education

Abstract

This research aimed to examine higher education instructors’ performance assessment in determining the cut-off point by setting criteria on the Wright map from big data. It is followed by designing performance assessment standards and assessing their quality. A total of 603 instructors from a Thai public university were selected as participants. The researchers employed a design-based research method encompassing four phases: analyzing the results of the performance assessment, formulating the standards-setting appraisal, applying trial and quality inspection, and improving the standards-setting appraisal approach. Data were analyzed using the Rasch model and the Maximum Likelihood Estimation method. The results of the determination of the cut-off point in terms of assessing instructors’ performance indicated that there are four cut-off points in ascending order, specifically, -11.67, -2.68, 4.59, and 9.76. The standards-setting appraisal showed that the assessment criteria consisted of five score ranges converted from estimation competency parameters into the scale and raw scores, respectively. Even though the standards-setting appraisal was determined, the researchers found that the transition point with regard to determination will be accurate and consistent in terms of those instructors who are at a moderate to high competency level and not suitable for evaluating those at a low competency level. The standards-setting appraisal approach is relevant for use as a criterion for recruiting and selecting higher education instructors. It can also support the development of sustainable human capital. It implies that instructors must possess high core competencies to match the high demand for quality teaching.