Evaluating Patient and Otolaryngologist Dialogues Generated by ChatGPT, Are They Adequate?

Abstract

Abstract AI applications are becoming more and more prevalent each day. ChatGPT is a recent AI tool that has amazed many people with its capabilities. It is expected that large language model solutions like ChatGPT will provide unique solutions and transform many industries. In many medical educational institutions, it is desired that medical students experience simulated patient encounters before meeting with real patients. These simulations can be designed to closely mimic the experience of a real-life patient encounter, allowing students to practice communication and history-taking skills in a realistic setting. Designing dialogues for these simulations is an important and time-consuming challenge. In this study, we evaluate if ChatGPT, an AI tool based on GPT-3, can generate adequate patient-doctor dialogues that can be utilized for medical student training. We analyze patient-doctor dialogues generated by ChatGPT for ten common ENT diseases and discuss the pros and cons of these dialogues. We believe the patient-doctor dialogues provided by ChatGPT can be a good starting point for teaching medical students how to communicate with patients.