Developing a Machine Learning Model to Predict In-hospital Mortality in Sepsis-complicated Acute Pancreatitis: A Retrospective Study Utilizing MIMIC-III, MIMIC-IV and eICU Databases (Preprint)

Abstract

Acute pancreatitis, a potentially lethal ailment that sees a surge in mortality rates when sepsis onset occurs, is quite prevalent. Given its severity, early intervention and tailored treatment become pivotal--underscored by the urgent need to anticipate mortality in patients plagued by acute pancreatitis complicated by sepsis. This highlights the crucial demand for the development of a cogent, interpretable mortality prediction model.

The objective of this study is to establish and validate a machine learning predictive model for determining the risk of in-hospital mortality in patients with acute pancreatitis (AP) complicated by sepsis. AP is a prevalent and potentially fatal inflammatory condition of the pancreas. When sepsis complicates AP, the likelihood of multiple organ dysfunction syndrome (MODS) increases, resulting in a very poor prognosis.

This study utilized patient cohorts from the MIMIC-III, MIMIC-IV, and eICU databases to conduct a regression analysis for both model development and validation. Feature selection was performed using the LASSO method and feature importance based on SHAP values, resulting in the establishment of 11 different machine learning predictive models. A stacked ensemble learning predictive model was developed using the Stacking ensemble algorithm. Model performance was evaluated using various metrics, including areas under the receiver operating characteristic (ROC) curves (AUC), PR curves, accuracy, recall, and F1 scores. The models were explained using the Kernel-SHAP algorithm and LIME algorithm.

An ensemble learning predictive model named AQlearner was developed based on feature variables screened via SHAP importance. The AUC value of AQlearner not only exceeded that of other individual models but also was higher than any machine learning predictive model established on variables screened out via LASSO, with an AUC reaching up to 0.873. Other performance indicators also showed excellent results. In the external validation cohort, the model's performance remained outstanding and was eventually selected. The Kernel-SHAP algorithm and LIME algorithm were used to interpret the machine learning models, and the results showed that this approach is feasible.

This study aimed to construct a machine-learning prediction model to evaluate the risk of in-hospital mortality in patients with acute pancreatitis complicated by sepsis. In clinical practice, feature selection based on the SHAP algorithm outperforms that based on Lasso. Furthermore, our ensemble learning predictive model exhibits superior performance compared to traditional single machine learning predictive models. The interpretability of machine learning models can be enhanced through the Kernel-SHAP algorithm and LIME algorithm. This SHAP feature importance-based model is highly suitable for clinical practice, as it can assist physicians in comprehending the underlying causes of predicted death and evaluating the in-hospital outcomes of critically ill patients.