Abstract
Objective: To compare the diagnostic accuracy of an artificial intelligence chatbot and clinical experts in managing vertigo-related diseases and evaluate the ability of the AI chatbot to address vertigo-related issues.
Methods: 20 clinical questions about vertigo were input into ChatGPT-4o, and three otologists evaluated the responses using a 5-point Likert scale for accuracy, comprehensiveness, clarity, practicality, and credibility. Readability was assessed using Flesch Reading Ease and Flesch-Kincaid Grade Level formulas. The model and two otologists diagnosed 15 outpatient vertigo cases, and their diagnostic accuracy was calculated. Statistical analysis used ANOVA and paired t-tests.
Results: ChatGPT-4o scored highest in credibility (4.78). Repeated Measures ANOVA showed significant differences across dimensions (F=2.682, p=0.038). Readability analysis revealed higher difficulty in diagnostic texts. The model's diagnostic accuracy was comparable to a clinician with one year of experience but inferior to a clinician with five years of experience (p=0.04).
Conclusion: ChatGPT-4o shows promise as a supplementary tool for managing vertigo but requires improvements in readability and diagnostic capabilities.