Patients with floaters: Answers from virtual assistants and large language models

Abstract

Objectives “Floaters,” a common complaint among patients of all ages, was used as a query term because it affects 30% of all people searching for eye care. The American Academy of Ophthalmology website's “floaters” section was used as a source for questions and answers ( www.aao.org ). Floaters is a visual obstruction that moves with the movement of the eye. They can be associated with retinal detachment, which can lead to vision loss. With the advent of large language model (LLM) chatbots ChatGPT, Bard versus virtual assistants (VA), Google Assistant, and Alexa, we analyzed their responses to “floaters.” Methods Using AAO.org, “Public & Patients,” and its related subsection, “EyeHealth A-Z”: Floaters and Flashes link, we asked four questions: (1) What are floaters? (2) What are flashes? (3) Flashes and Migraines? (4) Floaters and Flashes Treatment? to ChatGPT, Bard, Google Assistant, and Alexa. The American Academy of Ophthalmology (AAO) keywords were identified if they were highlighted. The “Flesch-Kincaid Grade Level” formula approved by the U.S. Department of Education, was used to evaluate the reading comprehension level for the responses. Results Of the chatbots and virtual assistants, Google Assistant is the only one that uses the term “ophthalmologist.” There is no mention of the urgency or emergency nature of floaters. AAO.org shows a lower reading level vs the LLMs and VA ( p = .11). The reading comprehension levels of ChatGPT, Bard, Google Assistant, and Alexa are higher (12.3, 9.7, 13.1, 8.1 grade) vs the AAO.org (7.3 grade). There is a higher word count for LLMs vs VA ( p < .0286). Conclusion Currently, ChatGPT, Bard, Google Assistant, and Alexa are similar. Factual information is present but all miss the urgency of the diagnosis of a retinal detachment. Translational relevance: Both the LLM and virtual assistants are free and our patients will use them to obtain “floaters” information. There may be errors of omission with ChatGPT and a lack of urgency to seek a physician's care.