Validation of an ICD-Code-Based Case Definition for Psychotic Illness Across Three Health Systems

Abstract

Abstract Background and Hypothesis Psychosis-associated diagnostic codes are increasingly being utilized as case definitions for electronic health record (EHR)-based algorithms to predict and detect psychosis. However, data on the validity of psychosis-related diagnostic codes is limited. We evaluated the positive predictive value (PPV) of International Classification of Diseases (ICD) codes for psychosis. Study Design Using EHRs at 3 health systems, ICD codes comprising primary psychotic disorders and mood disorders with psychosis were grouped into 5 higher-order groups. 1133 records were sampled for chart review using the full EHR. PPVs (the probability of chart-confirmed psychosis given ICD psychosis codes) were calculated across multiple treatment settings. Study Results PPVs across all diagnostic groups and hospital systems exceeded 70%: Mass General Brigham 0.72 [95% CI 0.68–0.77], Boston Children’s Hospital 0.80 [0.75–0.84], and Boston Medical Center 0.83 [0.79–0.86]. Schizoaffective disorder PPVs were consistently the highest across sites (0.80–0.92) and major depressive disorder with psychosis were the most variable (0.57–0.79). To determine if the first documented code captured first-episode psychosis (FEP), we excluded cases with prior chart evidence of a diagnosis of or treatment for a psychotic illness, yielding substantially lower PPVs (0.08–0.62). Conclusions We found that the first documented psychosis diagnostic code accurately captured true episodes of psychosis but was a poor index of FEP. These data have important implications for the case definitions used in the development of risk prediction models designed to predict or detect undiagnosed psychosis.