ChatGPT versus clinician: challenging the diagnostic capabilities of artificial intelligence in dermatology

Abstract

Abstract Background ChatGPT is an online language-based platform designed to answer questions in a human-like way, using deep learning ­technology. Objectives To examine the diagnostic capabilities of ChatGPT using real-world anonymized medical dermatology cases. Methods Clinical information from 90 consecutive patients referred to a single dermatology emergency clinic between June and December 2022 were examined. Thirty-six patients were included. Anonymized clinical information was transcribed and input into ChatGPT 4.0 followed by the question ‘What is the most likely diagnosis?’ The suggested diagnosis made by ChatGPT was then compared with the diagnosis made by dermatology. Results After inputting clinical history and examination data obtained by a dermatologist, ChatGPT made a correct primary diagnosis 56% of the time (n = 20). Using the clinical history and cutaneous signs recorded by nonspecialists, it was able to make a correct diagnosis 39% of the time (n = 14). This was similar to the diagnostic rate of nonspecialists (36%; n = 13), but it was much lower than that of dermatologists (83%; n = 30). There was no differential offered by referring sources 28% of the time (n = 10), unlike ChatGPT, which provided a differential diagnosis 100% of the time. Qualitative analysis showed that ChatGPT offered responses with caution, often justifying its reasoning. Conclusions This study illustrates that while ChatGPT has a diagnostic capability, in its current form it does not significantly improve the diagnostic yield in primary or secondary care.