Identifying the Presence, Activity, and Status of Extraintestinal Manifestations of Inflammatory Bowel Disease Using Natural Language Processing of Clinical Notes

Abstract

Abstract Background Extraintestinal manifestations (EIMs) occur commonly in inflammatory bowel disease (IBD), but population-level understanding of EIM behavior is difficult. We present a natural language processing (NLP) system designed to identify both the presence and status of EIMs using clinical notes from patients with IBD. Methods In a single-center retrospective study, clinical outpatient electronic documents were collected in patients with IBD. An NLP EIM detection pipeline was designed to determine general and specific symptomatic EIM activity status descriptions using Python 3.6. Accuracy, sensitivity, and specificity, and agreement using Cohen’s kappa coefficient were used to compare NLP-inferred EIM status to human documentation labels. Results The 1240 individuals identified as having at least 1 EIM consisted of 54.4% arthritis, 17.2% ocular, and 17.0% psoriasiform EIMs. Agreement between reviewers on EIM status was very good across all EIMs (κ = 0.74; 95% confidence interval [CI], 0.70-0.78). The automated NLP pipeline determining general EIM activity status had an accuracy, sensitivity, specificity, and agreement of 94.1%, 0.92, 0.95, and κ = 0.76 (95% CI, 0.74-0.79), respectively. Comparatively, prediction of EIM status using administrative codes had a poor sensitivity, specificity, and agreement with human reviewers of 0.32, 0.83, and κ = 0.26 (95% CI, 0.20-0.32), respectively. Conclusions NLP methods can both detect and infer the activity status of EIMs using the medical document an information source. Though source document variation and ambiguity present challenges, NLP offers exciting possibilities for population-based research and decision support in IBD.