Performance of a Novel Medical Artificial Intelligence Large Model (MedGo) on Supporting Decision-Making for Emergency Patients with Suspected Sepsis (Preprint)

Abstract

Large Artificial Intelligence (AI) language models have been increasingly applied in the medical field for disease prediction, diagnosis, and evaluation. However, research on AI-assisted early sepsis identification and screening remains scarce. Here, we conduct a retrospective study to evaluate the diagnostic efficacy of a novel medical large language model-MedGo developed by our collaborating team and us in early sepsis in emergency department (ED).

This study aims to evaluate the performance of a novel medical artificial intelligence large language model, MedGo, in supporting clinical decision-making for emergency department patients with suspected sepsis, specifically focusing on its diagnostic accuracy, comprehensiveness, readability, and analytical capabilities compared to physicians with varying levels of experience.

We retrospectively collected medical history data from 203 eligible patients treated at a tertiary teaching hospital between January 1, 2023 and January 1, 2024. MedGo’s performance was compared to that of junior and senior ED physicians across nine assessment tasks related to the diagnosis and management of sepsis . A five-point Likert scale was used to assess the four dimensions of accuracy, comprehensiveness, readability and case analysis skills.

MedGo exhibited diagnostic performance comparable to senior doctors, scoring 4 on the Likert Scale for accuracy, comprehensiveness, readability, and analytical capability, significantly surpassing junior doctors. Furthermore, MedGo's decision support enhanced both junior and senior doctors' diagnostic abilities, with junior doctors' performance equal that of seniors. Notably, MedGo consistently delivered exceptional results in diagnosing early sepsis cases of varying severity.

MedGo demonstrates remarkable diagnostic efficacy in early sepsis, effectively supporting clinicians of diverse experience levels in making informed decisions in the time-urgent ED. Although we acknowledge its limitations and emphasize the importance of comprehensive, standardized, systematic, and visualized medical history data in future research endeavors, the results underscore the potential of MedGo as a supportive tool in ED settings, thereby laying the groundwork for future developing specialized sepsis models.