Evaluating Real-Time Syndromic Surveillance Signals from Ambulatory Care Data in Four States

Abstract

Objectives. We evaluated a real-time ambulatory care-based syndromic surveillance system in four metropolitan areas of the United States. Methods. Health-care organizations and health departments in California, Massachusetts, Minnesota, and Texas participated during 2007–2008. Syndromes were defined using International Classification of Diseases, Ninth Revision diagnostic codes in electronic medical records. Health-care organizations transmitted daily counts of new episodes of illness by syndrome, date, and patient zip code. A space-time permutation scan statistic was used to detect unusual clustering. Health departments followed up on e-mailed alerts. Distinct sets of related alerts (“signals”) were compared with known outbreaks or clusters found using traditional surveillance. Results. The 62 alerts generated corresponded to 17 distinct signals of a potential outbreak. The signals had a median of eight cases (range: 3–106), seven zip code areas (range: 1–88), and seven days (range: 3–14). Two signals resulted from true clusters of varicella; six were plausible but unconfirmed indications of disease clusters, six were considered spurious, and three were not investigated. The median investigation time per signal by health departments was 50 minutes (range: 0–8 hours). Traditional surveillance picked up 124 clusters of illness in the same period, with a median of six ill per cluster (range: 2–75). None was related to syndromic signals. Conclusions. The system was able to detect two true clusters of illness, but none was of public health interest. Possibly due to limited population coverage, the system did not detect any of 124 known clusters, many of which were small. The number of false alarms was reasonable.