Development and external validation of machine learning-based models to predict patients with cellulitis developing sepsis during hospitalisation

Abstract

ObjectiveCellulitis is the most common cause of skin-related hospitalisations, and the mortality of patients with sepsis remains high. Some stratification models have been developed, but their performance in external validation has been unsatisfactory. This study was designed to develop and compare different models for predicting patients with cellulitis developing sepsis during hospitalisation.DesignThis is a retrospective cohort study.SettingThis study included both the development and the external-validation phases from two independent large cohorts internationally.Participants and methodsA total of 6695 patients with cellulitis in the Medical Information Mart for Intensive care (MIMIC)-IV database were used to develop models with different machine-learning algorithms. The best models were selected and then externally validated in 2506 patients with cellulitis from the YiduCloud database of our university. The performances and robustness of selected models were further compared in the external-validation group by area under the curve (AUC), diagnostic accuracy, sensitivity, specificity and diagnostic OR.Primary outcome measuresThe primary outcome of interest in this study was the development based on the Sepsis-3.0 criteria during hospitalisation.ResultsPatient characteristics were significantly different between the two groups. In internal validation, XGBoost was the best model, with an AUC of 0.780, and AdaBoost was the worst model, with an AUC of 0.585. In external validation, the AUC of the artificial neural network (ANN) model was the highest, 0.830, while the AUC of the logistic regression (LR) model was the lowest, 0.792. The AUC values changed less in the boosting and ANN models than in the LR model when variables were deleted.ConclusionsBoosting and neural network models performed slightly better than the LR model and were more robust in complex clinical situations. The results could provide a tool for clinicians to detect hospitalised patients with cellulitis developing sepsis early.