Validating the Emergency Department Avoidability Classification (EDAC): A cluster randomized single-blinded agreement study

Abstract

Introduction The Emergency Department Avoidability Classification (EDAC) retrospectively classifies emergency department (ED) visits that could have been safely managed in subacute primary care settings, but has not been validated against a criterion standard. A validated EDAC could enable accurate and reliable quantification of avoidable ED visits. We compared agreement between the EDAC and ED physician judgements to specify avoidable ED visits. Materials and methods We conducted a cluster randomized, single-blinded agreement study in an academic hospital in Hamilton, Canada. ED visits between January 1, 2019, and December 31, 2019 were clustered based on EDAC classes and randomly sampled evenly. A total of 160 ED visit charts were randomly assigned to ten participating ED physicians at the academic hospital for evaluation. Physicians judged if the ED visit could have been managed appropriately in subacute primary care (an avoidable visit); each ED visit was evaluated by two physicians independently. We measured interrater agreement between physicians with a Cohen’s kappa and 95% confidence intervals (CI). We evaluated the correlation between the EDAC and physician judgements using a Spearman rank correlation and ordinal logistic regression with odds ratios (ORs) and 95% CIs. We examined the EDAC’s precision to identify avoidable ED visits using accuracy, sensitivity and specificity. Results ED physicians agreed on 139 visits (86.9%) with a kappa of 0.69 (95% CI 0.59–0.79), indicating substantial agreement. Physicians judged 96.2% of ED visits classified as avoidable by the EDAC as suitable for management in subacute primary care. We found a high correlation between the EDAC and physician judgements (0.64), as well as a very strong association to classify avoidable ED visits (OR 80.0, 95% CI 17.1–374.9). The EDACs avoidable and potentially avoidable classes demonstrated strong accuracy to identify ED visits suitable for management in subacute care (82.8%, 95% CI 78.2–86.8). Discussion The EDAC demonstrated strong evidence of criterion validity to classify avoidable ED visits. This classification has important potential for accurately monitoring trends in avoidable ED utilization, measuring proportions of ED volume attributed to avoidable visits and informing interventions intended at reducing ED use by patients who do not require emergency or life-saving healthcare.