Abstract
ChatGPT (OpenAI) has introduced a vision version recently, indicating its potential application in interpreting radiological images. Mitral regurgitation (MR) is the most common valvular heart abnormality, no study has attempted to evaluate the severity of MR using ChatGPT. In the present study, we aimed to explore the clinical potential of ChatGPT vision for MR assessment in transthoracic echocardiography. In this retrospective study, 293 color Doppler images, including 75 mild, 113 moderate, and 105 severe MR, were submitted to ChatGPT 4o with a prompt to assess the severity of MR. Each image was submitted 3 times to collect 3 answers to assess the consistency of ChatGPT’s responses with the first answer used for the confusion matrix and assessment of ChatGPT’s performance in predicting mild, moderate, and severe MR. ChatGPT 4o demonstrated relatively low performance with an overall accuracy of 45.4%. Prediction of moderate and severe MR achieved better performance, with a sensitivity of 62.8%, specificity of 47.2%, and balanced accuracy of 55.0% for moderate MR, and a sensitivity of 58.1%, specificity of 68.1%, and balanced accuracy of 63.1% for severe MR. While performance for mild MR was worse, with sensitivity of only 1.3%, although specificity of 97.7% and balanced accuracy of 49.5%. ChatGPT 4o showed potential but underperformed in assessment of MR severity. Further studies are needed to assess the vision capability of large language models as a potential tool for interpretation of radiology images.