Informing the development of a scoring system for National Health Service Clinical Impact Awards; a Delphi process and simulated scoring exercise

Abstract

Objectives To establish principles informing a new scoring system for the UK's Clinical Impact Awards and pilot a system based on those principles. Design A three-round online Delphi process was used to generate consensus from experts on principles a scoring system should follow. We conducted a shadow scoring exercise of 20 anonymised, historic applications using a new scoring system incorporating those principles. Setting Assessment of clinical excellence awards for senior doctors and dentists in England and Wales. Participants The Delphi panel comprised 45 members including clinical excellence award assessors and representatives of professional bodies. The shadow scoring exercise was completed by 24 current clinical excellence award assessors. Main outcome measures The Delphi panel rated the appropriateness of a series of items. In the shadow scoring exercise, a novel scoring system was used with each of five domains rated on a 0–10 scale. Results Consensus was achieved around principles that could underpin a future scoring system; in particular, a 0–10 scale with the lowest point on the scale reflecting someone operating below the expectations of their job plan was agreed as appropriate. The shadow scoring exercise showed similar levels of reliability between the novel scoring system and that used historically, but with potentially better distinguishing performance at higher levels of performance. Conclusions Clinical excellence awards represent substantial public spending and thus far the deployment of these funds has lacked a strong evidence base. We have developed a new scoring system in a robust manner which shows improvements over current arrangements.