Evaluating the validity of the selection measures used for the UK’s foundation medical training programme: a national cohort study

Abstract

ObjectivesCurrently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria.MethodsData were obtained from the UK Medical Education Database (UKMED) on 14 131 UK applicants to the foundation programme starting in 2013 and 2014. These data included training outcomes in the form of Annual Reviews of Competency Progression (ARCPs), which indicated whether the programme was successfully completed. The relationship between applicants’ performance on the three selection measures to the odds of successful programme completion were modelled.ResultsOn univariable analyses, all three measures were associated with the odds of successful completion of the programme. Converting the SJT score to deciles to compare the effect sizes suggested that one decile increase in the EPM increased the odds of completing the programme by approximately 15%, whereas the equivalent value was 8% for the SJT scores. On multivariable analyses (with all three measures included in the model), these effects were only independently and statistically significant for EPM decile (OR 1.14, 95% CI 1.10 to 1.18, p<0.001) and SJT z-score decile (OR 1.05, 95% CI 1.01 to 1.09, p=0.02).ConclusionsThe EPM decile and SJT scores may be effective selection measures for the foundation programme. However, educational achievements does not add value to the other two measures when predicting programme completion. Thus, its usefulness in this context is less clear. Moreover, our findings suggest that the weighting for the EPM decile score, relative to SJT performance, should be increased.