Development and validation of machine learning-driven prediction model for serious bacterial infection among febrile children in emergency departments

Abstract

Serious bacterial infection (SBI) in children, such as bacterial meningitis or sepsis, is an important condition that can lead to fatal outcomes. Therefore, since it is very important to accurately diagnose SBI, SBI prediction tools such as ‘Refined Lab-score’ or ‘clinical prediction rule’ have been developed and used. However, these tools can predict SBI only when there are values of all factors used in the tool, and if even one of them is missing, the tools become useless. Therefore, the purpose of this study was to develop and validate a machine learning-driven model to predict SBIs among febrile children, even with missing values. This was a multicenter retrospective observational study including febrile children <6 years of age who visited Emergency departments (EDs) of 3 different tertiary hospitals from 2016 to 2018. The SBI prediction model was trained with a derivation cohort (data from two hospitals) and externally tested with a validation cohort (data from a third hospital). A total of 11,973 and 2,858 patient records were included in the derivation and validation cohorts, respectively. In the derivation cohort, the area under the receiver operating characteristic curve (AUROC) of the RF model was 0.964 (95% confidence interval [CI], 0.943–0.986), and the area under the precision-recall curve (AUPRC) was 0.753 (95% CI, 0.681–0.824). The conventional LR (CLR) model showed corresponding values of 0.902 (95% CI, 0.894–0.910) and 0.573 (95% CI, 0.560–0.586), respectively. In the validation cohort, the AUROC (95% CI) of the RF model was 0.950 (95% CI, 0.945–0.956), the AUPRC was 0.605 (95% CI, 0.593–0.616), and the CLR presented corresponding values of 0.815 (95% CI, 0.789–0.841) and 0.586 (95% CI, 0.553–0.619), respectively. We developed a machine learning-driven prediction model for SBI among febrile children, which works robustly despite missing values. And it showed superior performance compared to CLR in both internal validation and external validation.