Automated Identification of Clinical Procedures in Free-Text Electronic Clinical Records with a Low-Code Named Entity Recognition Workflow

Abstract

Abstract Background Clinical procedures are often performed in outpatient clinics without prior scheduling at the administrative level, and documentation of the procedure often occurs solely in free-text clinical electronic notes. Natural language processing (NLP), particularly named entity recognition (NER), may provide a solution to extracting procedure data from free-text electronic notes. Methods Free-text notes from outpatient ophthalmology visits were collected from the electronic clinical records at a single institution over 3 months. The Prodigy low-code annotation tool was used to create an annotation dataset and train a custom NER model for clinical procedures. Clinical procedures were extracted from the entire set of clinical notes. Results There were a total of 5,098 clinic notes extracted for the study period; 1,923 clinic notes were used to build the NER model, which included a total of 231 manual annotations. The NER model achieved an F-score of 0.767, a precision of 0.810, and a recall of 0.729. The most common procedures performed included intravitreal injections of therapeutic substances, removal of corneal foreign bodies, and epithelial debridement of corneal ulcers. Conclusion The use of a low-code annotation software tool allows the rapid creation of a custom annotation dataset to train a NER model to identify clinical procedures stored in free-text electronic clinical notes. This enables clinicians to rapidly gather previously unidentified procedural data for quality improvement and auditing purposes. Low-code annotation tools may reduce time and coding barriers to clinician participation in NLP research.