Related Experiment Video
Updated: Jan 12, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Artificial intelligence in patient education: evaluating large language models for understanding rheumatology
Claudia Mendoza-Pinto1,2, Pamela Munguía-Realpozo1,2, Ivet Etchegaray-Morales2
1Unidad de Enfermedades Reumáticas y Autoinmunes Sistémicas, Unidad Médica de Alta Especialidad- Centro de Investigación Biomédica de Oriente, Instituto Mexicano del Seguro Social, Puebla, México.
Background:
Inadequate health literacy hinders positive health outcomes, yet medical literature often exceeds the general population's comprehension level. While health authorities recommend patient materials be at a sixth-grade reading level, scientific articles typically require college-level proficiency. Large language models (LLMs) like ChatGPT show potential for simplifying complex text, possibly bridging this gap.
Objective:
This study evaluated the effectiveness of ChatGPT 4.0 in enhancing the readability of peer-reviewed rheumatology articles for layperson comprehension.
Methods:
Twelve open-access rheumatology articles authored by the senior investigators were included. Baseline readability was evaluated utilizing Flesch-Kincaid Grade Level (FKGL) and Simple Measure of Gobbledygook (SMOG) indices. Each article was processed by ChatGPT 4.0 with a prompt requesting simplification to a sixth-grade level. Two expert rheumatologists evaluated the generated summaries' appropriateness (accuracy, absence of errors/omissions). Readability changes were analyzed using paired t-tests.
Results:
ChatGPT significantly improved readability (P < .0001), reducing the average reading level from approximately 15th grade (FKGL: 15.06, SMOG: 14.08) to 10th grade (FKGL: 10.52, SMOG: 9.48). The expert reviewers deemed the generated summaries appropriate and accurate. The average word count was significantly reduced from 3,517 to 446 words (P = 0.047).
Conclusions:
ChatGPT effectively lowered the reading complexity of specialized rheumatology literature, making it more accessible than the original publications. However, the achieved 10th-grade reading level still exceeds the recommended sixth-grade level for patient education materials. While LLMs are a promising tool, their output may require further refinement or expert review to meet optimal health literacy standards and ensure equitable patient understanding in rheumatology.
