Identifying secondary findings in PET/CT reports in oncological cases: A quantifying study using automated Natural Language Processing

Abstract

AbstractBackgroundBecause of their accuracy, positron emission tomography/computed tomography (PET/CT) examinations are ideally suited for the identification of secondary findings but there are only few quantitative studies on the frequency and number of those.Most radiology reports are freehand written and thus secondary findings are not presented as structured evaluable information and the effort to manually extract them reliably is a challenge. Thus we report on the use of natural language processing (NLP) to identify secondary findings from PET/CT conclusions.Methods4,680 anonymized German PET/CT radiology conclusions of five major primary tumor entities were included in this study. Using a commercially available NLP tool, secondary findings were annotated in an automated approach. The performance of the algorithm in classifying primary diagnoses was evaluated by statistical comparison to the ground truth as recorded in the patient registry. Accuracy of automated classification of secondary findings within the written conclusions was assessed in comparison to a subset of manually evaluated conclusions.ResultsThe NLP method was evaluated twice. First, to detect the previously known principal diagnosis, with an F1 score between 0.65 and 0.95 among 5 different principal diagnoses.Second, affirmed and speculated secondary diagnoses were annotated, and the error rate of false positives and false negatives was evaluated. Overall, rates of false-positive findings (1.0%-5.8%) and misclassification (0%-1.1%) were low compared with the overall rate of annotated diagnoses. Error rates for false-negative annotations ranged from 6.1% to 24%. More often, several secondary findings were not fully captured in a conclusion. This error rate ranged from 6.8% to 45.5%.ConclusionsNLP technology can be used to analyze unstructured medical data efficiently and quickly from radiological conclusions, despite the complexity of human language. In the given use case, secondary findings were reliably found in in PET/CT conclusions from different main diagnoses.