Postgraduate medical education quality metrics panels can be enhanced by including learner outcomes

Abstract

Postgraduate medical education (PME) quality assurance at Health Education England (HEE) currently relies upon survey data. As no one metric can reflect all aspects of training, and each has its limitations, additional metrics should be explored. At HEE (West Midlands), we explored the use of learner outcomes, speciality examination pass rates and Annual Review of Competence Progression (ARCP) outcomes, as quality metrics. Feedback received from our local Quality Forum of 40 senior educators frames the discussion through this paper. Overall, learner outcomes are useful quality metrics that add to survey data to provide a more comprehensive picture of PME quality. However, the utility of ARCP outcomes as quality metrics is currently limited by concerns regarding variations in ARCP practice between regions. To address these concerns, ARCPs need the same processes, rigour, scrutiny and investment as other high-stakes assessments. This will improve the reliability and validity of the ARCP as an assessment and improve the usefulness of ARCP outcomes as quality metrics. Research is required to determine the optimal combination of metrics to use in PME quality assurance and to appraise the validity and reliability of the ARCP as an assessment.