Coding of Childhood Psychiatric and Neurodevelopmental Disorders in Electronic Health Records of a Large Integrated Healthcare System: A Validation Study (Preprint)

Abstract

Mental, emotional, and behavioral disorders are chronic pediatric conditions, and their prevalence has been on the rise over recent decades. Affected children have long-term health sequelae and a decline in health-related quality of life. Due to the lack of a validated database for pharmacoepidemiological research on selected mental, emotional, and behavioral disorders, there is uncertainty in the reported prevalence in the literature.

To evaluate the accuracy of pediatric mental, emotional, and behavioral disorders-related coding in a large integrated healthcare system’s electronic health records (EHR) and compare the coding quality before and after implementation of the International Classification of Diseases-Clinical Modification [ICD]-10-CM coding and pre- and post-COVID-19 pandemic era.

Medical records of 1200 member children aged 2-17 years with at least one clinical visit during pre-COVID-19 (01/01/2012 - 12/31/2014 [ICD-9-CM coding period] and 01/01/2017 - 12/31/2019 [ICD-10-CM coding period]) and post-COVID-19 (01/01/2021 - 12/31/2022) pandemic era were selected with stratified random sampling from EHR for chart review. Two trained Research Associates (RA) reviewed EHR for all potential cases of autism spectrum disorders (ASD), attention-deficit hyperactivity disorder (ADHD), major depression disorder (MDD), anxiety disorders (AD), and disruptive behavior disorders (DBD) in children during the study period. Children were considered cases only if there was a mention of any one of the conditions (Yes for diagnosis) in the electronic chart during the corresponding time period. The validity of diagnosis codes was evaluated by directly comparing it with the gold standard of chart abstraction using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), the summary statistics of F-score (F), and Youden’s J Statistic (J). Kappa statistic (K) for interrater reliability (IRR) among the two abstractors was calculated.

The overall agreement between the identification of mental, behavioral, and emotional conditions using diagnosis codes compared to medical record abstraction was strong and similar across the ICD-9-CM and ICD-10-CM coding periods as well as during the pre- pandemic and pandemic time periods. Performance of AD coding, while strong, was relatively lower compared to the other conditions. The weighted sensitivity, specificity, PPV and NPV for each of the 5 conditions were as follows: ASD (100.0%, 100.0%, 99.2%, 100.0%), ADHD (100.0%, 99.9%, 99.2%, 100.0%), DBD (100.0%, 100.0%, 100.0%, 100.0%), AD (87.7%, 100.0%, 100.0%, 99.2%) and MDD (100.0%, 100.0%, 99.2%, 100.0%). The F-score and Youden’s J Statistic ranged between 87.7 and 100.0%. The overall agreement between abstractors was almost perfect (K=95%).

Diagnostic codes are quite reliable for identifying selected childhood mental, behavioral, and emotional conditions. The findings remained similar during the pandemic and after the implementation of the ICD-10-CM coding in the EHR system.

Not Applicable