Performance of ChatGPT in classifying periodontitis according to the 2018 classification of periodontal diseases

Abstract

Abstract Objectives This study assessed the ability of ChatGPT, an artificial intelligence(AI) language model, to determine the stage, grade, and extent of periodontitis based on the 2018 classification. Materials and methods This study used baseline digital data of 200 untreated periodontitis patients to compare standardized reference diagnoses (RDs) with ChatGPT findings and determine the best criteria for assessing stage and grade. RDs were provided by four experts who examined each case. Standardized texts containing the relevant information for each situation were constructed to query ChatGPT. RDs were compared to ChatGPT's responses. Variables influencing the responses of ChatGPT were evaluated. Results ChatGPT successfully identified the periodontitis stage, grade, and extent in 59.5%, 50.5%, and 84.0% of cases, respectively. Cohen’s kappa values for stage, grade and extent were respectively 0.447, 0.284, and 0.652. A multiple correspondence analysis showed high variance between ChatGPT’s staging and the variables affecting the stage (64.08%) and low variance between ChatGPT’s grading and the variables affecting the grade (42.71%). Conclusions The present performance of ChatGPT in the classification of periodontitis exhibited a reasonable level. However, it is expected that additional improvements would increase its effectiveness and broaden its range of functionalities (NCT05926999). Clinical relevance Despite ChatGPT's current limitations in accurately classifying periodontitis, it is important to note that the model has not been specifically trained for this task. However, it is expected that with additional improvements, the effectiveness and capabilities of ChatGPT might be enhanced.