Quantifying Healthcare Provider Perceptions of a Novel Deep Learning Algorithm to Predict Sepsis: Electronic Survey (Preprint)

Abstract

Sepsis is a major cause of morbidity and mortality for which early intervention improves patient outcomes. However, many patients experience delays in appropriate diagnosis and treatment. Predictive modeling and artificial intelligence may aid in early recognition of sepsis but there remains a considerable disconnect between the development of predictive algorithms and their use in clinical care. Despite the importance of user experience for the adoption and efficacy of clinical predictive models, there are relatively few studies focused on provider acceptance and feedback on the deployment of such models.

Evaluate healthcare worker perception and acceptance of a deep learning model for the prediction of sepsis in the ED

COMPOSER, a previously described deep learning algorithm in use at two EDs of a large academic medical center, utilizes routinely collected vital-signs, laboratory results, demographics, medications, and comorbidities to predict sepsis prior to clear clinical presentation. When the COMPOSER risk score crosses a predefined detection threshold, a Best Practice Advisory (BPA) is triggered for nursing staff. For physicians and advanced practice providers (APP), COMPOSER alerts are displayed on the ED trackboard. An internally developed and validated survey in accordance with the CHERRIES checklist was distributed to a convenience sample of team members taking care of a patient for whom a COMPOSER alert crossed the pre-defined threshold. Recruitment occurred between May and September 2023 and was administered using a vendor survey tool.

A total of 114 responses were received with 76 from MD/DOs, 34 from RNs, and 4 from NP/PAs. 53% of respondents were from providers with less than 5 years of experience. 77% of respondents had a positive or neutral perception of the alert’s usefulness. Providers with 0-5 years of experience were more likely to have increased expectation that a patient had sepsis after the alert (p=.021), and were more likely to say that the alert was useful in care of a patient (p=.016), than those with 6+ years of experience. Finally, physicians with 0-5 years of experience were more likely to say that the alert changed their management of the patient (p=.048) than physicians with 6+ years of experience.

Less experienced providers and nurses were more likely to perceive benefit from the alert, and the alerts were overall received favorably. Future clinical AI model implementations might consider focused alert patterns and education to improve reception and reduce fatigue.