Source Characteristics Influence AI-Enabled Orthopaedic Text Simplification: Recommendations for the Future

Area of Science:

• Artificial Intelligence
• Health Informatics
• Medical Education

Background:

• Orthopaedic patient education materials (PEMs) often contain complex language, hindering patient comprehension.
• Simplifying these materials is crucial for improving health literacy and patient outcomes.
• Large language models (LLMs) offer a potential solution for text simplification.

Purpose of the Study:

• To assess the effectiveness of various LLMs in simplifying orthopaedic PEMs.
• To identify factors predicting successful text transformation by LLMs.
• To evaluate the impact of LLMs on the readability of orthopaedic patient materials.

Main Methods:

• Forty-eight orthopaedic PEMs were transformed using GPT-4, GPT-3.5, Claude 2, and Llama 2.
• Readability was measured using Flesch-Kincaid Reading Ease (FKRE) and Grade Level (FKGL) scores before and after transformation.
• Statistical and machine learning methods analyzed text characteristics and their correlation with transformation success.

Main Results:

• All tested LLMs significantly improved FKRE and FKGL scores (p < 0.01).
• GPT-4 demonstrated superior performance, achieving a mean FKGL of 6.72 ± 0.99.
• Transformation success was influenced by original text features like word length and sentence complexity, varying by LLM.

Conclusions:

• LLMs are effective tools for simplifying complex orthopaedic PEMs, enhancing readability.
• GPT-4 exhibited the most significant improvements in text readability.
• Initial text characteristics are critical predictors of LLM transformation success, informing AI-driven health literacy strategies.