Detection of patients with COVID-19 by the emergency medical services in Lombardy through an operator-based interview and machine learning models

Abstract

BackgroundThe regional emergency medical service (EMS) in Lombardy (Italy) developed clinical algorithms based on operator-based interviews to detect patients with COVID-19 and refer them to the most appropriate hospitals. Machine learning (ML)-based models using additional clinical and geospatial epidemiological data may improve the identification of infected patients and guide EMS in detecting COVID-19 cases before confirmation with SARS-CoV-2 reverse transcriptase PCR (rtPCR).MethodsThis was an observational, retrospective cohort study using data from October 2020 to July 2021 (training set) and October 2021 to December 2021 (validation set) from patients who underwent a SARS-CoV-2 rtPCR test within 7 days of an EMS call. The performance of an operator-based interview using close contact history and signs/symptoms of COVID-19 was assessed in the training set for its ability to determine which patients had an rtPCR in the 7 days before or after the call. The interview accuracy was compared with four supervised ML models to predict positivity for SARS-CoV-2 within 7 days using readily available prehospital data retrieved from both training and validation sets.ResultsThe training set includes 264 976 patients, median age 74 (IQR 55–84). Test characteristics for the detection of COVID-19-positive patients of the operator-based interview were: sensitivity 85.5%, specificity 58.7%, positive predictive value (PPV) 37.5% and negative predictive value (NPV) 93.3%. Contact history, fever and cough showed the highest association with SARS-CoV-2 infection. In the validation set (103 336 patients, median age 73 (IQR 50–84)), the best-performing ML model had an AUC of 0.85 (95% CI 0.84 to 0.86), sensitivity 91.4% (95 CI% 0.91 to 0.92), specificity 44.2% (95% CI 0.44 to 0.45) and accuracy 85% (95% CI 0.84 to 0.85). PPV and NPV were 13.3% (95% CI 0.13 to 0.14) and 98.2% (95% CI 0.98 to 0.98), respectively. Contact history, fever, call geographical distribution and cough were the most important variables in determining the outcome.ConclusionML-based models might help EMS identify patients with SARS-CoV-2 infection, and in guiding EMS allocation of hospital resources based on prespecified criteria.