A Comparison of Machine Learning Methods and Conventional Logistic Regression for the Prediction of In-Hospital Mortality in Acute Biliary Pancreatitis

Abstract

Objectives For population databases, multivariable regressions are established analytical standards. The utilization of machine learning (ML) in population databases is novel. We compared conventional statistical methods and ML for predicting mortality in biliary acute pancreatitis (biliary AP). Methods Using the Nationwide Readmission Database (2010–2014), we identified patients (age ≥18 years) with admissions for biliary AP. These data were randomly divided into a training (70%) and test set (30%), stratified by the outcome of mortality. The accuracy of ML and logistic regression models in predicting mortality was compared using 3 different assessments. Results Among 97,027 hospitalizations for biliary AP, mortality rate was 0.97% (n = 944). Predictors of mortality included severe AP, sepsis, increasing age, and nonperformance of cholecystectomy. Assessment metrics for predicting the outcome of mortality, the scaled Brier score (odds ratio [OR], 0.24; 95% confidence interval [CI], 0.16–0.33 vs 0.18; 95% CI, 0.09–0.27), F-measure (OR, 43.4; 95% CI, 38.3–48.6 vs 40.6; 95% CI, 35.7–45.5), and the area under the receiver operating characteristic (OR, 0.96; 95% CI, 0.94–0.97 vs 0.95; 95% CI, 0.94–0.96) were comparable between the ML and logistic regression models, respectively. Conclusions For population databases, traditional multivariable analysis is noninferior to ML-based algorithms in predictive modeling of hospital outcomes for biliary AP.