Extensive validation and prospective observation of the impact of an AI-based rapid antibiotics susceptibility prediction platform in multiple medical centers

Abstract

AbstractImportanceNo medical artificial intelligence (AI) has been robustly validated and deployed in a clinical laboratory in real-world settings, and the clinical impact of the medical AI remains unknown.ObjectiveTo deploy a medical AI platform for rapid antibiotics susceptibility test (AST) prediction, and evaluate its clinical impacts.DesignA medical AI platform, XBugHunter, was extensively validated (internal validation, time-wise validation, and independent testing) with data between May 22, 2013 and June 30, 2019. The clinical impact was evaluated based on a prospective observation from February 1 to September 30, 2020 during deployment.SettingData was collected in two tertiary medical centers in Taiwan, and the AI was deployed in a tertiary medical center.ParticipantsFor validation, 90,064 consecutive cases were included. During the deployment, a prospective observational cohort of 1,490 consecutive cases was collected.ExposuresAST prediction from XBugHunterMain outcomes and MeasuresDiagnostic performance of XBugHunter was evaluated during validation. The clinical impact was evaluated in terms of the saving of inappropriate antibiotics prescription, AST turn-around-time, and mortality of bacteremia during deployment.ResultsPredictive models consistently performed well in the extensive validations. In the deployment, XBugHunter’s predictive sensitivity and specificity for Staphylococcus aureus (oxacillin) were 0.95 (95% CI, 0.82–0.98) and 0.97 (95% CI, 0.94–0.99), respectively. For Acinetobacter baumannii (multiple drugs), the sensitivity was 0.95 (95% CI, 0.91–0.99) and specificity was 0.93 (95% CI, 0.88–0.98). The turn-around-time reduction on reporting AST of blood cultures was 35.72 h (standard deviation: 15.55 h). Death within 28 days occurred in 28 of 162 S. aureus bacteremia patients (17.28%) in the XBugHunter intervention group, which was lower than the 28 days’ mortality rate (28.06% [55/196]) in the same period of time in 2019, without XBugHunter. The relative risk reduction was 38.4% (relative risk, 0.62; 95% CI, 0.41–0.92). Regarding antibiotic prescriptions, 2723.7 defined daily dose per year of inappropriate antibiotics could be avoided for treating S. aureus by deploying XBugHunter.Conclusions and RelevanceAmong S. aureus bacteremia patients, this study demonstrated that XBugHunter can prevent inappropriate antibiotic use, and the adjustment of antibiotic treatment can yield lower mortality.Key PointsQuestionWhat is the clinical impact of XBugHunter, a machine learning-based, antibiotic susceptibility test prediction platform?FindingsIn the prospective observational cohort of deploying XBugHunter, the reduction of turn-around-time of reporting antibiotic susceptibility test was 35.72h. The reduction of S. aureus bacteremia mortality rate was 10.78%, and the estimated saving of inappropriate antibiotics uses was 2723.7 defined daily dose per year.MeaningDeployment of XBugHunter provides a more rapid report of antibiotic susceptibility test, and thus reduces inappropriate antibiotics prescription and mortality of S. aureus bacteremia.