Systematic review of methodological quality of individual performance measurement in surgery

Abstract

Abstract Background Calls for greater transparency with improved quality, safety and outcomes have led to performance tracking of individual surgeons. This study evaluated the methodology of studies investigating individual performance in surgery. Methods MEDLINE, Embase, PsycINFO, AMED and the Cochrane Database of Systematic Reviews (from their inception to July 2014) were searched. Two authors independently reviewed citations using predetermined inclusion and exclusion criteria; 91 data points per study were extracted. Results The search strategy yielded 8514 citations; 101 were eligible, comprising 1 006 037 procedures by 14 455 surgeons. Thirty-four studies were prospective and 66 were retrospective. The aim of the studies was either to assess individual performance and describe the learning curve of a procedure, to describe factors influencing performance, or to describe methods for routine performance monitoring. Some 51·5 per cent of the studies investigated 500 or fewer procedures. Most (77 of 101) were single-centre studies. Less than half of the studies (42, 41·6 per cent) employed statistical modelling or stratification to adjust performance measures. Forty studies (39·6 per cent) adjusted outcomes for case mix. Seventeen (16·8 per cent) adjusted metrics for surgeon-specific factors. Thirteen studies (12·9 per cent) considered clustering in their analyses. The most frequent outcome studied was duration of operation (59·4 per cent), followed by complication rate (45·5 per cent) and reoperation rate (29·7 per cent); 15·8 per cent of studies recorded mortality, and 4·0 per cent explored patient satisfaction. Only 48·5 per cent of studies displayed procedural learning curves using a graph. Conclusion There exist substantial shortcomings in methodological quality, outcome measurements and quality improvement evaluation among current studies of individual surgical performance. Methodological guidelines should be established to ensure that assessments are valid.