A time-adjusted control chart for monitoring surgical outcome variations

Abstract

Background Statistical Process Control (SPC) tools providing feedback to surgical teams can improve patient outcomes over time. However, the quality of routinely available hospital data used to build these tools does not permit full capture of the influence of patient case-mix. We aimed to demonstrate the value of considering time-related variables in addition to patient case-mix for detection of special cause variations when monitoring surgical outcomes with control charts. Methods A retrospective analysis from the French nationwide hospital database of 151,588 patients aged 18 and older admitted for colorectal surgery between January 1st, 2014, and December 31st, 2018. GEE multilevel logistic regression models were fitted from the training dataset to predict surgical outcomes (in-patient mortality, intensive care stay and reoperation within 30-day of procedure) and applied on the testing dataset to build control charts. Surgical outcomes were adjusted on patient case-mix only for the classical chart, and additionally on secular (yearly) and seasonal (quarterly) trends for the enhanced control chart. The detection of special cause variations was compared between those charts using the Cohen’s Kappa agreement statistic, as well as sensitivity and positive predictive value with the enhanced chart as the reference. Results Within the 5-years monitoring period, 18.9% (28/148) of hospitals detected at least one special cause variation using the classical chart and 19.6% (29/148) using the enhanced chart. 59 special cause variations were detected overall, among which 19 (32.2%) discordances were observed between classical and enhanced charts. The observed Kappa agreement between those charts was 0.89 (95% Confidence Interval [95% CI], 0.78 to 1.00) for detecting mortality variations, 0.83 (95% CI, 0.70 to 0.96) for intensive care stay and 0.67 (95% CI, 0.46 to 0.87) for reoperation. Depending on surgical outcomes, the sensitivity of classical versus enhanced charts in detecting special causes variations ranged from 0.75 to 0.89 and the positive predictive value from 0.60 to 0.89. Conclusion Seasonal and secular trends can be controlled as potential confounders to improve signal detection in surgical outcomes monitoring over time.