Comparison of machine learning models to predict long-term outcomes after severe traumatic brain injury

Abstract

OBJECTIVE An estimated 1.5 million people die every year worldwide from traumatic brain injury (TBI). Physicians are relatively poor at predicting long-term outcomes early in patients with severe TBI. Machine learning (ML) has shown promise at improving prediction models across a variety of neurological diseases. The authors sought to explore the following: 1) how various ML models performed compared to standard logistic regression techniques, and 2) if properly calibrated ML models could accurately predict outcomes up to 2 years posttrauma. METHODS A secondary analysis of a prospectively collected database of patients with severe TBI treated at a single level 1 trauma center between November 2002 and December 2018 was performed. Neurological outcomes were assessed at 3, 6, 12, and 24 months postinjury with the Glasgow Outcome Scale. The authors used ML models including support vector machine, neural network, decision tree, and naïve Bayes models to predict outcome across all 4 time points by using clinical information available on admission, and they compared performance to a logistic regression model. The authors attempted to predict unfavorable versus favorable outcomes (Glasgow Outcome Scale scores of 1–3 vs 4–5), as well as mortality. Models’ performance was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC) with 95% confidence interval and balanced accuracy. RESULTS Of the 599 patients in the database, the authors included 501, 537, 469, and 395 at 3, 6, 12, and 24 months posttrauma. Across all time points, the AUCs ranged from 0.71 to 0.85 for mortality and from 0.62 to 0.82 for unfavorable outcomes with various modeling strategies. Decision tree models performed worse than all other modeling approaches for multiple time points regarding both unfavorable outcomes and mortality. There were no statistically significant differences between any other models. After proper calibration, the models had little variation (0.02–0.05) across various time points. CONCLUSIONS The ML models tested herein performed with equivalent success compared with logistic regression techniques for prognostication in TBI. The TBI prognostication models could predict outcomes beyond 6 months, out to 2 years postinjury.