Investigating the impact of structured reporting on the linguistic standardization of radiology reports through natural language processing over a 10-year period

Abstract

Abstract Objectives To investigate how a transition from free text to structured reporting affects reporting language with regard to standardization and distinguishability. Methods A total of 747,393 radiology reports dictated between January 2011 and June 2020 were retrospectively analyzed. The body and cardiothoracic imaging divisions introduced a reporting concept using standardized language and structured reporting templates in January 2016. Reports were segmented by a natural language processing algorithm and converted into a 20-dimension document vector. For analysis, dimensionality was reduced to a 2D visualization with t-distributed stochastic neighbor embedding and matched with metadata. Linguistic standardization was assessed by comparing distinct report types’ vector spreads (e.g., run-off MR angiography) between reporting standards. Changes in report type distinguishability (e.g., CT abdomen/pelvis vs. MR abdomen) were measured by comparing the distance between their centroids. Results Structured reports showed lower document vector spread (thus higher linguistic similarity) compared with free-text reports overall (21.9 [free-text] vs. 15.9 [structured]; − 27.4%; p < 0.001) and for most report types, e.g., run-off MR angiography (15.2 vs. 1.8; − 88.2%; p < 0.001) or double-rule-out CT (26.8 vs. 10.0; − 62.7%; p < 0.001). No changes were observed for reports continued to be written in free text, e.g., CT head reports (33.2 vs. 33.1; − 0.3%; p = 1). Distances between the report types’ centroids increased with structured reporting (thus better linguistic distinguishability) overall (27.3 vs. 54.4; + 99.3 ± 98.4%) and for specific report types, e.g., CT abdomen/pelvis vs. MR abdomen (13.7 vs. 37.2; + 171.5%). Conclusion Structured reporting and the use of factual language yield more homogenous and standardized radiology reports on a linguistic level, tailored to specific reporting scenarios and imaging studies. Clinical relevance Information transmission to referring physicians, as well as automated report assessment and content extraction in big data analyses, may benefit from standardized reporting, due to consistent report organization and terminology used for pathologies and normal findings. Key Points • Natural language processing and t-distributed stochastic neighbor embedding can transform radiology reports into numeric vectors, allowing the quantification of their linguistic standardization. • Structured reporting substantially increases reports’ linguistic standardization (mean: − 27.4% in vector spread) and distinguishability (mean: + 99.3 ± 98.4% increase in vector distance) compared with free-text reports. • Higher standardization and homogeneity outline potential benefits of structured reporting for information transmission and big data analyses.