Validation of an Automated System for Identifying Complications of Serious Pediatric Emergencies

Abstract

BACKGROUND Illness complications are condition-specific adverse outcomes. Detecting complications of pediatric illness in administrative data would facilitate widespread quality measurement, however the accuracy of such detection is unclear. METHODS We conducted a cross-sectional study of patients visiting a large pediatric emergency department. We analyzed those <22 years old from 2012 to 2019 with 1 of 14 serious conditions: appendicitis, bacterial meningitis, diabetic ketoacidosis (DKA), empyema, encephalitis, intussusception, mastoiditis, myocarditis, orbital cellulitis, ovarian torsion, sepsis, septic arthritis, stroke, and testicular torsion. We applied a method using disposition, diagnosis codes, and procedure codes to identify complications. The automated determination was compared with the criterion standard of manual health record review by using positive predictive values (PPVs) and negative predictive values (NPVs). Interrater reliability of manual reviews used a κ. RESULTS We analyzed 1534 encounters. PPVs and NPVs for complications were >80% for 8 of 14 conditions: appendicitis, bacterial meningitis, intussusception, mastoiditis, myocarditis, orbital cellulitis, sepsis, and testicular torsion. Lower PPVs for complications were observed for DKA (57%), empyema (53%), encephalitis (78%), ovarian torsion (21%), and septic arthritis (64%). A lower NPV was observed in stroke (68%). The κ between reviewers was 0.88. CONCLUSIONS An automated method to measure complications by using administrative data can detect complications in appendicitis, bacterial meningitis, intussusception, mastoiditis, myocarditis, orbital cellulitis, sepsis, and testicular torsion. For DKA, empyema, encephalitis, ovarian torsion, septic arthritis, and stroke, the tool may be used to screen for complicated cases that may subsequently undergo manual review.