BACKGROUND
The increasing use of Chat Generative Pre-Trained Transformer (ChatGPT) in medical education and clinical practice necessitates evaluating its reliability, particularly in geriatrics.
OBJECTIVE
This study aimed to evaluate ChatGPT's trustworthiness in geriatrics through three distinct approaches: evaluating ChatGPT’s geriatric attitude, knowledge, and its application to two geriatric syndrome vignettes of polypharmacy and falls in older adults
METHODS
We used the validated UCLA geriatrics attitude instrument to evaluate ChatGPT's geriatric attitude and the UCLA geriatrics knowledge tests to evaluate ChatGPT’s geriatrics knowledge and compared ChatGPT's responses and performance with that of medical students, residents, and geriatrics fellows. We also evaluate ChatGPT's application to two geriatric vignettes of polypharmacy and falls in older adults.
RESULTS
ChatGPT demonstrated less age-biased outputs to the UCLA geriatrics attitude instrument compared to medical students and residents including a more positive and less negative attitude towards older adults. On the UCLA geriatric knowledge tests, ChatGPT outperformed medical students and first-year residents and showed comparable performance to second- and third-year residents but lower than geriatrics medicine fellows. In practical applications to two geriatric syndrome vignettes, ChatGPT provided appropriate recommendations for polypharmacy management, accurately identifying most of potentially inappropriate medications, drug-drug and drug-disease interactions, though it missed a few of drug-disease interactions and produced a few of ChatGPT hallucinations. ChatGPT identified most fall risks and provided comprehensive fall prevention recommendations for an older adult.
CONCLUSIONS
This study suggests that ChatGPT can be a valuable supplemental tool in geriatrics, offering reliable information with less age-bias, robust geriatric knowledge, and comprehensive recommendations for managing two common geriatric syndromes (polypharmacy and falls) that are consistent with the evidence from the guidelines, systematic review, and other types of studies. ChatGPT's potential as an educational and clinical resource could significantly benefit early-stage trainees, healthcare providers, and laypersons. Further research with ChatGPT-4 and larger geriatric question sets, and more geriatric syndromes is needed to expand and confirm these findings before adopting ChatGPT widely to geriatric education and practice.
CLINICALTRIAL
n/a