Large language model may assist diagnosis of SAPHO syndrome by bone scintigraphy

Abstract

Abstract Objective: In this study, we employed a large language model to evaluate the diagnostic efficacy of radiology reports of bone scintigraphy in the context of identifying SAPHO syndrome, and further examined the potential of such a model to augment the diagnostic procedure. Methods: Imaging data and clinical information of 151 patients (105/46 women/men, mean age: 53.5 years) who underwent bone scintigraphy for suspected SAPHO syndrome between January 2007 and December 2022 were retrospectively reviewed. ChatGPT-4.0 was used as the large language model. The diagnostic performance of the large language model was verified by comparing the cases judged to have SAPHO syndrome that fulfilled Kahn’s classification criteria based on a combination of concise radiology reports and skin lesions such as palmoplantar pustulosis, with cases diagnosed with SAPHO syndrome by rheumatologists based on all clinical information. The diagnostic performance of the large language model was verified. Results: The diagnostic accuracy of a large language model for analyzing bone scintigraphy radiology reports in conjunction with information about skin symptoms, such as palmoplantar pustulosis, achieved a sensitivity of 83.5%, specificity of 69.4%, and an overall accuracy of 76.8%. Discussion: While this research is an initial endeavor dedicated to the utilization of a substantial language model in the creation of a database for imaging diagnostics of rheumatic conditions, it exhibits commendable diagnostic accuracy, particularly for diseases with a wide range of symptoms like SAPHO syndrome, indicating a positive outlook for subsequent studies. Conclusion: This research indicates the prospective value of extensive language models in scrutinizing radiology accounts from bone scintigraphy for the diagnosis of SAPHO syndrome.