Leveraging Large Language Models to Improve the Readability of German Online Medical Texts: Evaluation Study

Area of Science:

• Medical Informatics
• Natural Language Processing
• Health Literacy

Background:

• Online patient education materials (PEMs) are often too complex for the general public, hindering comprehension and informed health decisions.
• Large language models (LLMs) present a potential solution for simplifying complex medical texts.
• The effectiveness of LLMs in simplifying German-language PEMs remains largely unexplored.

Purpose of the Study:

• To evaluate the capability of LLMs to enhance the readability of German online medical texts.
• To determine if LLM-simplified texts meet recommended readability levels, such as an eighth-grade reading level.

Main Methods:

• Sixty German online medical texts were selected for simplification using four LLMs: ChatGPT-3.5, ChatGPT-4o, Microsoft Copilot, and Le Chat.
• Readability was assessed using the Flesch Reading Ease (FRE) and Wiener Sachtextformel (WSTF) scores before and after LLM simplification.
• Statistical analysis, including Student's t-tests for paired samples, was employed to compare readability scores and assess significant reductions.

Main Results:

• Original texts were generally difficult to read (average WSTF 11.24, FRE 35.92).
• ChatGPT-3.5, ChatGPT-4o, and Microsoft Copilot significantly improved text readability (average WSTF 9.96, 10.6, and 8.99 respectively; FRE 45.04, 39.23, and 49.0 respectively).
• Despite improvements, LLM simplification did not consistently achieve scores below the eighth-grade reading level across all models.

Conclusions:

• LLMs can moderately improve the readability of German patient education materials, enhancing accessibility for online readers.
• These findings highlight the potential of LLMs to bridge the health literacy gap by making medical information more understandable.
• This study provides initial evidence on the efficacy of LLMs for simplifying German online medical texts.