Affiliation:
1. Department of Otolaryngology–Head and Neck Surgery The Third Affiliated Hospital of Sun Yat‐Sen University Guangzhou China
2. Department of Allergy The Third Affiliated Hospital of Sun Yat‐Sen University Guangzhou China
3. Naso‐Orbital‐Maxilla and Skull Base Center The Third Affiliated Hospital of Sun Yat‐Sen University Guangzhou China
4. Key Laboratory of Airway Inflammatory Disease Research and Innovative Technology Translation Guangzhou China
Abstract
AbstractObjectiveThis study aims to evaluate the accuracy of ChatGPT in answering allergic rhinitis (AR) and chronic rhinosinusitis (CRS) related questions.Study DesignThis is a cross‐sectional study.SettingEach question was inputted as a separate, independent prompt.MethodsResponses to AR (n = 189) and CRS (n = 242) related questions, generated by GPT‐3.5 and GPT‐4, were independently graded for accuracy by 2 senior rhinology professors, with disagreements adjudicated by a third reviewer.ResultsOverall, ChatGPT demonstrated a satisfactory performance, accurately answering over 80% of questions across all categories. Specifically, GPT‐4.0's accuracy in responding to AR‐related questions significantly exceeded that of GPT‐3.5, but distinction not evident in CRS‐related questions. Patient‐originated questions had a significantly higher accuracy compared to doctor‐originated questions when utilizing GPT‐4.0 to respond to AR‐related questions. This discrepancy was not observed with GPT‐3.5 or in the context of CRS‐related questions. Across different types of content, ChatGPT excelled in covering basic knowledge, prevention, and emotion for AR and CRS. However, it experienced challenges when addressing questions about recent advancements, a trend consistent across both GPT‐3.5 and GPT‐4.0 iterations. Importantly, the accuracy of responses remained unaffected when questions were posed in Chinese.ConclusionOur findings suggest ChatGPT's capability to convey accurate information for AR and CRS patients, and offer insights into its performance across various domains, guiding its utilization and improvement.