BACKGROUND
In the context of a deepening global shortage of health workers, and particularly the COVID-19 pandemic, there is growing international interest in and use of online symptom checkers (OSCs). However, the evidence surrounding the safety and accuracy of OSCs remains inconclusive so far. The triage and diagnostic accuracy of these tools is an essential aspect that needs to be addressed before pushing any further implementation.
OBJECTIVE
This systematic review aimed to summarise the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns.
METHODS
Searches were conducted in Medline, Embase, CINAHL, HMIC and Web of Science, as well as the citations of the studies selected for full-text screening. We included peer-reviewed studies published in English between 1 January 2010 and 17 February 2022 with a controlled and quantitative assessment of triage and/or diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health professionals, and disease- or speciality-specific OSCs. Screening and data extraction were carried out independently by two reviewers for each study. We performed a descriptive narrative synthesis.
RESULTS
21,284 studies were screened and 14 were included. These used either clinical vignettes, medical records or direct input by patients. Six studies reported on both triage and diagnostic accuracy, seven focused on triage accuracy, and one on diagnostic accuracy. These outcomes were assessed based on the diagnostic and/or triage recommendation attached to the vignette in the case of vignette studies, and/or those provided by nurses or GPs, including through face-to-face and phone consultations. Both diagnostic and triage accuracy varied greatly between OSCs. Overall diagnostic accuracy was deemed to be low, and almost always lower than the comparator. Similarly, most studies showed suboptimal triage accuracy overall, with a few notable exceptions. The main variables affecting the levels of diagnostic and triage accuracy were the severity and urgency of the condition, the use of AI algorithms and demographic questions. However, the impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. Included studies had either a moderate or high risk of bias according to the revised tool for the Quality Assessment of Diagnostic Accuracy Studies 2.
CONCLUSIONS
While OSCs have significant potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy prior to wide-scale adoption in community and healthcare settings. Future studies should aim to use a common methodology and/or agreed standard for evaluation to facilitate objective benchmarking and validation.
CLINICALTRIAL