Assessing the Competence of ChatGPT-3.5 Artificial Intelligence System in Executing the ACLS Protocol of the AHA 2020

Abstract

Abstract Objectives: Artificial intelligence (AI) has become the focus of current studies, particularly due to its contribution in preventing human labor and time loss. The most important contribution of AI applications in the medical field will be to provide opportunities for increasing clinicians' gains, reducing costs, and improving public health. This study aims to assess the proficiency of ChatGPT-3.5, one of the most advanced AI applications available today, in its knowledge of current information based on the American Heart Association (AHA) 2020 guidelines. Methods: An 80-question quiz in a question-and-answer format, which includes the current AHA 2020 application steps, was prepared and applied to ChatGPT-3.5 in both English (ChatGPT-3.5 English) and native language (ChatGPT-3.5 Turkish) versions in March 2023. The questions were prepared only in the native language for emergency medicine specialists. Results: We found a similar success rate of over 80% in all questions asked to ChatGPT-3.5 and two independent emergency medicine specialists with at least 5 years of experience who did not know each other. ChatGPT-3.5 achieved a 100% success rate in all questions related to the General Overview for Current AHA Guideline, Airway Management, and Ventilation chapters in English. Conclusions: Our study indicates that ChatGPT-3.5 provides similar accurate and up-to-date responses as experienced emergency specialists in the AHA 2020 Advanced Cardiac Life Support Guidelines. This suggests that with future updated versions of ChatGPT, instant access to accurate and up-to-date information based on textbooks and guidelines will be possible.