A comparison of the assessments used in campus-based years at the College of Medicine, Imam Abdulrahman Bin Faisal University, Saudi Arabia

Abstract

Abstract Study Purpose Multiple assessment tools are used to assess future doctors’ knowledge, clinical skills, and professional attitudes. In the present research, the difficulty level and discriminating ability of different types of written and performance-based assessments designed to measure the knowledge and competency of medical students were compared. Methods The assessment data of 2nd & 3rd-year medical students (the academic year 2020-2021) in the College of Medicine at Imam Abdulrahman Bin Faisal University (IAU) were retrospectively reviewed. Based on end-of-the-year overall grades, students were divided into high and low scorers. Both groups were compared by independent sample t-test for their mean scores achieved in each type of assessment. Difficulty level and discriminating ability of the assessments were also explored. MS Excel and Statistical Package for Social Sciences (SPSS version 27) were used for analysis. Area under the curve was calculated through ROC analysis. A p-value <0.05 was believed significant. Results In each type of written assessment, the high scorer group achieved significantly higher scores compared to the low scorers. Among performance-based assignments (except the PBLs), scores did not differ significantly between high and low scorers. The difficulty level of performance-based assessments was “easy” whereas it was “moderate” for written assessments (except the OSCE). The discriminating ability of performance-based assessments was “poor” whereas it was “moderate/excellent” for written assessments (except the OSCE). Conclusion Our study results indicate that written assessments have excellent discriminatory ability. However, performance-based assessments are not as difficult and discriminatory as written assessments. The PBLs are relatively discriminatory among all performance-based assessments. Key messages What is already known on this topic At Imam Abdulrahman Bin Faisal University, written and performance-based assessments both are graded on criterion-referenced scales. The student’s grades at the end of the year are an aggregate of his/her scores in written and performance-based assessments. What this study adds Our study results show that performance-based assessments are not as difficult and discriminatory in differentiating between high and low scorers as written assessments. How this study might affect research, practice or policy Performance-based assessments should be made a hurdle exam (pass or fail) for the students to move to the next level, or students must pass each assessment component (written and performance-based) separately.