Evaluation of a Novel Content-Based Image Retrieval System for the Differentiation of Interstitial Lung Diseases in CT Examinations

Abstract

To evaluate the reader’s diagnostic performance against the ground truth with and without the help of a novel content-based image retrieval system (CBIR) that retrieves images with similar CT patterns from a database of 79 different interstitial lung diseases. We evaluated three novice readers’ and three resident physicians‘ (with at least three years of experience) diagnostic performance evaluating 50 different CTs featuring 10 different patterns (e.g., honeycombing, tree-in bud, ground glass, bronchiectasis, etc.) and 24 different diseases (sarcoidosis, UIP, NSIP, Aspergillosis, COVID-19 pneumonia etc.). The participants read the cases first without assistance (and without feedback regarding correctness), and with a 2-month interval in a random order with the assistance of the novel CBIR. To invoke the CBIR, a ROI is placed into the pathologic pattern by the reader and the system retrieves diseases with similar patterns. To further narrow the differential diagnosis, the readers can consult an integrated textbook and have the possibility of selecting high-level semantic features representing clinical information (chronic, infectious, smoking status, etc.). We analyzed readers’ accuracy without and with CBIR assistance and further tested the hypothesis that the CBIR would help to improve diagnostic performance utilizing Wilcoxon signed rank test. The novice readers demonstrated an unassisted accuracy of 18/28/44%, and an assisted accuracy of 84/82/90%, respectively. The resident physicians demonstrated an unassisted accuracy of 56/56/70%, and an assisted accuracy of 94/90/96%, respectively. For each reader, as well as overall, Sign test demonstrated statistically significant (p < 0.01) difference between the unassisted and the assisted reads. For students and physicians, Chi²-test and Mann-Whitney-U test demonstrated statistically significant (p < 0.01) difference for unassisted reads and statistically insignificant (p > 0.01) difference for assisted reads. The evaluated CBIR relying on pattern analysis and featuring the option to filter the results of the CBIR by predominant characteristics of the diseases via selecting high-level semantic features helped to drastically improve novices’ and resident physicians’ accuracy in diagnosing interstitial lung diseases in CT.