Effect of a Sepsis Prediction Algorithm on Patient Mortality, Length of Stay, and Readmission

Abstract

AbstractObjectiveTo validate performance of a machine learning algorithm for severe sepsis determination up to 48 hours before onset, and to evaluate the effect of the algorithm on in-hospital mortality, hospital length of stay, and 30-day readmission.SettingThis cohort study includes a combined retrospective analysis and clinical outcomes evaluation: a dataset containing 510,497 patient encounters from 461 United States health centers for retrospective analysis, and a multiyear, multicenter clinical data set of real-world data containing 75,147 patient encounters from nine hospitals for clinical outcomes evaluation.ParticipantsFor retrospective analysis, 270,438 adult patients with at least one documented measurement of five out of six vital sign measurements were included. For clinical outcomes analysis, 17,758 adult patients who met two or more Systemic Inflammatory Response Syndrome (SIRS) criteria at any point during their stay were included.ResultsAt severe sepsis onset, the MLA demonstrated an AUROC of 0.91 (95% CI 0.90, 0.92), which exceeded those of MEWS (0.71, P<001), SOFA (0.74; P<.001), and SIRS (0.62; P<.001). For severe sepsis prediction 48 hours in advance of onset, the MLA achieved an AUROC of 0.77 (95% CI 0.73, 0.80). For the clinical outcomes study, when using the MLA, hospitals saw an average 39.5% reduction of in-hospital mortality, a 32.3% reduction in hospital length of stay, and a 22.7% reduction in 30-day readmission rate.ConclusionsThe MLA accurately predicts severe sepsis onset up to 48 hours in advance using only readily available vital signs in retrospective validation. Reductions of in-hospital mortality, hospital length of stay, and 30-day readmissions were observed in real-world clinical use of the MLA. Results suggest this system may improve severe sepsis detection and patient outcomes over the use of rules-based sepsis detection systems.KEY POINTSQuestionIs a machine learning algorithm capable of accurate severe sepsis prediction, and does its clinical implementation improve patient mortality rates, hospital length of stay, and 30-day readmission rates?FindingsIn a retrospective analysis that included datasets containing a total of 585,644 patient encounters from 461 hospitals, the machine learning algorithm demonstrated an AUROC of 0.93 at time of severe sepsis onset, which exceeded those of MEWS (0.71), SOFA (0.74), and SIRS (0.62); and an AUROC of 0.77 for severe sepsis prediction 48 hours in advance of onset. In an analysis of real-world data from nine hospitals across 75,147 patient encounters, use of the machine learning algorithm was associated with a 39.5% reduction in in-hospital mortality, a 32.3% reduction in hospital length of stay, and a 22.7% reduction in 30-day readmission rate.MeaningThe accurate and predictive nature of this algorithm may encourage early recognition of patients trending toward severe sepsis, and therefore improve sepsis related outcomes.STRENGTHS AND LIMITATIONS OF THIS STUDYA retrospective study of machine learning severe sepsis prediction from a dataset with 510,497 patient encounters demonstrates high accuracy up to 48 hours prior to onset.A multicenter clinical study of real-world data using this machine learning algorithm for severe sepsis alerts achieved reductions of in-hospital mortality, length of stay, and 30-day readmissions.The required presence of an ICD-9 code to classify a patient as severely septic in our retrospective analysis potentially limits our ability to accurately classify all patients.Only adults in US hospitals were included in this study.For the real-world section of the study, we cannot eliminate the possibility that implementation of a sepsis algorithm raised general awareness of sepsis within a hospital, which may lead to higher recognition of septic patients, independent of algorithm performance.