Translating ophthalmic medical jargon with artificial intelligence: a comparative comprehension study

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Ophthalmology Communication

Background:

• Technical jargon in ophthalmology notes hinders comprehension for allied health professionals.
• Effective interprofessional communication is crucial for patient care and safety.
• Large language models (LLMs) offer potential solutions for simplifying complex medical terminology.

Purpose of the Study:

• To evaluate the efficacy of ChatGPT-4.0 in translating technical ophthalmology terminology.
• To compare ChatGPT-4.0's translation performance against other leading LLMs.
• To assess the impact of LLM-generated translations on comprehension and clinical utility.

Main Methods:

• An observational cross-sectional study involving 15 clinical ophthalmology notes.
• Notes were translated using ChatGPT-4.0, ChatGPT-4o, Claude 3 Sonnet, and Google Gemini.
• Family physicians evaluated original and translated notes for comprehension and utility; readability was assessed using Flesch scores; ophthalmologists identified translation errors.

Main Results:

• LLM translations significantly improved comprehension (4.7/5.0) and usefulness (4.6/5.0) compared to original notes (3.7/5.0 and 3.8/5.0, respectively; p < 0.001).
• ChatGPT-4.0 was the most preferred model (8/15 cases), with ChatGPT-4o also showing strong performance.
• While all models had errors, ChatGPT-4.0 and 4o exhibited fewer inaccuracies; translated notes showed slightly increased linguistic complexity.

Conclusions:

• ChatGPT-4.0 effectively enhances the comprehensibility of ophthalmology notes, improving interprofessional communication.
• LLMs show promise for medical translation, but require careful implementation and ongoing refinement.
• Further research is needed to validate LLM utility across diverse medical specialties and languages.