Patients with floaters: Answers from virtual assistants and large language models.

Area of Science:

• Ophthalmology and Artificial Intelligence
• Digital Health and Patient Information Dissemination

Background:

• Eye floaters are a common visual disturbance affecting a significant portion of the population seeking eye care information.
• The increasing use of AI chatbots and virtual assistants necessitates an evaluation of their accuracy and completeness in providing health information.
• Floaters can be associated with serious conditions like retinal detachment, requiring timely medical intervention.

Purpose of the Study:

• To analyze and compare the information provided by leading AI chatbots (ChatGPT, Bard) and virtual assistants (Google Assistant, Alexa) regarding eye floaters.
• To assess the reading comprehension level and keyword usage of AI responses compared to a reputable source like the American Academy of Ophthalmology (AAO.org).

Main Methods:

• Utilized the American Academy of Ophthalmology (AAO.org) 'Floaters and Flashes' section as a reference for four key questions.
• Queried ChatGPT, Bard, Google Assistant, and Alexa with these questions to gather their responses.
• Evaluated responses for factual accuracy, keyword inclusion (e.g., 'ophthalmologist'), and reading comprehension using the Flesch-Kincaid Grade Level formula.

Main Results:

• AI responses generally contained factual information but lacked emphasis on the urgency of diagnosing conditions like retinal detachment.
• Google Assistant was the only AI that mentioned 'ophthalmologist'.
• AI-generated content had a higher reading comprehension level (8.1-13.1) compared to AAO.org (7.3 grade) and a higher word count than virtual assistants.

Conclusions:

• While AI tools offer accessible information on floaters, they may omit critical details regarding the urgency of seeking professional medical evaluation.
• There is a risk of 'errors of omission,' potentially leading patients to underestimate the seriousness of floaters and delay seeking care from an ophthalmologist.
• The study highlights the need for AI to convey the potential urgency associated with eye floaters to ensure patient safety and timely diagnosis.