Improving Patient Safety Event Report Classification with Machine Learning and Contextual Text Representation

Abstract

Adverse events caused by medical errors pose a significant threat to patient safety, with estimates of 251,454 deaths and a cost of $17.1 billion to the healthcare system annually in the United States. Patient safety event (PSE) reports play a vital role in identifying measures to prevent adverse events, but their utility is dependent on the accurate classification of PSE reports. Recent studies have used static natural language processing (NLP) and machine learning (ML) techniques to automate PSE report classification. However, the use of static NLP has limitations in differentiating the meaning of words in disparate contexts, which can lead to inferior classification results. Thus, this study proposes to utilize contextual text representation produced from neural NLP methods to improve the accuracy of PSE report classification. The results suggest that the contextual text representation can further improve the performance of PSE classifiers. The best-performing classifier, a support vector machine trained with contextual text representation (Roberta-base) reaches an accuracy of 0.75 and a ROCAUC score of 0.94, surpassing all ML classifiers trained with static text representations. Furthermore, the confusion matrix of the best classifier exposes latent deficiencies in the PSE reports' classification taxonomy, such as the multi-class nature of PSE and conceptually related event types. The study's findings can save time for PSE reclassification, enhance the learning capabilities of the reporting system, ultimately improve patient safety