Standard-Setting Methods for Assessment in a Post-Graduate Medical College

Abstract

Context: Standard-setting procedures assess candidates’ competence in an examination. Different standard-setting methods produce different pass scores, and no gold standard exists currently. The quality of the standard-setting process is critical in medical examinations where true competency needs to be determined for safe medical practice. Aims: This study assessed the standard-setting methods the college uses to determine the pass scores in the various parts of the fellowship examinations and compared these methods with the arbitrary 50% previously used. Settings and Design: A cross-sectional comparative study to assess the standard-setting methods adopted, which was applied to the September/October/November 2023 fellowship examinations. Methods: This was a total population survey involving the 16 faculties of the College. Secondary data from a compilation of approved results was used. Data Analysis: Descriptive and analytical statistics in Microsoft Excel program. Results: The methods for standard-setting adopted by the college were assessed, and their mean pass scores were analysed. The mean pass score for the primary multiple choice questions examinations was 46.7%, lower than the previously used arbitrary 50% mark, and this difference was statistically significant (P < 0.05). The mean pass scores for the other examinations were higher than the previously used arbitrary 50% mark, but these differences were not statistically significant (P > 0.05). Conclusion: Implementation of the approved standard-setting methods can be improved with more training and retraining of faculties and examiners so that results that are consistent with their purpose, and that align well with other measures of competency can be produced.