Policy impact considerations for using an SJT to improve diversity in medical selection

Abstract

Abstract Introduction: Traditionally, medical selection has relied heavily on indicators of prior academic attainment. Recent studies have shown that situational judgement tests (SJTs) show promise in widening access to medical education. However, there has been little research regarding the weighting and sequencing of selection tools to improve the diversity of applicants throughout the process. This study extends previous research by measuring the magnitude of the policy change that would be required for an SJT to have substantial impact on the diversity of the shortlisting pool for interviews with a graduate entry medical program, when used in conjunction with the Graduate Medical Schools Admissions Test (GAMSAT).Method: A SJT was developed and piloted to rank applicants for interview offers (N=270). Five indicators of diversity were investigated to address gender, ethnicity and socio-economic status (SES). The impact on diversity was explored by weighting SJT scores with the current shortlisting tool (GAMSAT) using four weighting options (0%, 25%, 50%, 75%). Results: GAMSAT scores differed significantly (p<0.05) for all diversity indicators except ethnicity. SJT scores differed significantly (p<0.05) for two indicators; first in family to have a degree and/or applicants whose prior degree was not medical science related performed better. The odds ratios for interview shortlisting using 100% GAMSAT were significantly reduced with a 50% GAMSAT and SJT weighting. Discussion: This pilot study demonstrates the policy impact when incorporating an SJT into a traditional medical education selection process. It provides evidence for substantial policy changes in the selection process to improve diversity and widen access.