Affiliation:
1. Division of Cardiac Surgery, Bristol Heart Institute, Translational Health Sciences, University of Bristol , Bristol, UK
2. Alan Turing Institute , London, UK
3. Department of Statistics, University of Oxford , Oxford, UK
Abstract
Abstract
OBJECTIVES
To perform a systematic comparison of in-hospital mortality risk prediction post-cardiac surgery, between the predominant scoring system—European System for Cardiac Operative Risk Evaluation (EuroSCORE) II, logistic regression (LR) retrained on the same variables and alternative machine learning techniques (ML)—random forest (RF), neural networks (NN), XGBoost and weighted support vector machine.
METHODS
Retrospective analyses of prospectively routinely collected data on adult patients undergoing cardiac surgery in the UK from January 2012 to March 2019. Data were temporally split 70:30 into training and validation subsets. Mortality prediction models were created using the 18 variables of EuroSCORE II. Comparisons of discrimination, calibration and clinical utility were then conducted. Changes in model performance, variable-importance over time and hospital/operation-based model performance were also reviewed.
RESULTS
Of the 227 087 adults who underwent cardiac surgery during the study period, there were 6258 deaths (2.76%). In the testing cohort, there was an improvement in discrimination [XGBoost (95% confidence interval (CI) area under the receiver operator curve (AUC), 0.834–0.834, F1 score, 0.276–0.280) and RF (95% CI AUC, 0.833–0.834, F1, 0.277–0.281)] compared with EuroSCORE II (95% CI AUC, 0.817–0.818, F1, 0.243–0.245). There was no significant improvement in calibration with ML and retrained-LR compared to EuroSCORE II. However, EuroSCORE II overestimated risk across all deciles of risk and over time. The calibration drift was lowest in NN, XGBoost and RF compared with EuroSCORE II. Decision curve analysis showed XGBoost and RF to have greater net benefit than EuroSCORE II.
CONCLUSIONS
ML techniques showed some statistical improvements over retrained-LR and EuroSCORE II. The clinical impact of this improvement is modest at present. However the incorporation of additional risk factors in future studies may improve upon these findings and warrants further study.
Funder
British Heart Foundation-Turing research
Publisher
Oxford University Press (OUP)
Subject
Cardiology and Cardiovascular Medicine,Pulmonary and Respiratory Medicine,General Medicine,Surgery
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献