Evaluating ChatGPT responses to frequently asked patient questions regarding periprosthetic joint infection after total hip and knee arthroplasty

Abstract

Background Patients access relevant information concerning their orthopaedic surgery resources through multiple information channels before presenting for clinical treatment. Recently, artificial intelligence (AI)-powered chatbots have become another source of information for patients. The currently developed AI chat technology ChatGPT (OpenAI LP) is an application for such purposes and it has been rapidly gaining popularity, including for patient education. This study sought to evaluate whether ChatGPT can correctly answer frequently asked questions (FAQ) regarding periprosthetic joint infection (PJI). Methods Twelve FAQs about PJI after hip and knee arthroplasty were identified from the websites of fifteen international clinical expert centres. ChatGPT was confronted with these questions and its responses were analysed for their accuracy using an evidence-based approach by a multidisciplinary team. Responses were categorised in four groups: (1) Excellent response that did not require additional improvement; (2) Satisfactory responses that required a small amount of improvement; (3) Satisfactory responses that required moderate improvement; and (4) Unsatisfactory responses that required a large amount of improvement. Results From the analysis of the responses given by the chatbot, no reply received an ‘unsatisfactory’ rating; one did not require any correction; and the majority of the responses required low (7 out of 12) or moderate (4 out of 12) clarification. Although a few responses required minimal clarification, the chatbot responses were generally unbiased and evidence-based, even when asked controversial questions. Conclusions The AI-chatbot ChatGPT was able to effectively answer the FAQs of patients seeking information around PJI diagnosis and treatment. The given information was also written in a manner that can be assumed to be understandable by patients. The chatbot could be a valuable clinical tool for patient education and understanding around PJI treatment in the future. Further studies should evaluate its use and acceptance by patients with PJI.