ChatGPT as a tool to improve readability of rheumatology patient education materials: a positive start with...

Patient education materials are vital to health education and, according to the American Medical Association (AMA), should not be written above a 6th grade reading level (Weiss 2003). We aim to assess American College of Rheumatology (ACR) patient education readability and use ChatGPT 3.5 to adjust it to a 6th grade reading level.

Four validated readability assessment tests were selected to assess the readability levels for 96 online ACR patient education materials on rheumatic conditions and treatments (American College of Rheumatology: Diseases and Conditions n.d; American College of Rheumatology: Treatments n.d.) the Gunning Fog Index (GFI), Flesch-Kincaid Grade Level (FKGL), Coleman-Liau Index (CLI), and Simple Measure of Gobbledygook Index (SMOG). All education materials were inputted into ChatGPT with the prompt "Rewrite the following text at a 5th grade reading level:". These converted excerpts were evaluated for readability using the online readability scoring software (Readability Formulas. Readability Scoring System 14).

The mean (± SD) readability ratings of GFI, FKGL, CLI, and SMOG from the 96 original ACR patient education materials were 14.24 (1.85), 11.50 (1.74), 13.31 (1.42), and 10.33 (1.29), respectively. The ACR materials had a mean reading level of 12th grade (12.02 ± 1.35). After the ChatGPT prompt, the calculated average reading level for the simplified education materials ranged from grades 7 to 13, with a mean of 9.35 ± 1.12. The mean (± SD) readability ratings of GFI, FKGL, CLI, and SMOG for the simplified versions were 10.70 (1.39), 8.29 (1.21), 10.18 (1.11), and 7.68 (1.05), respectively.

The readability of ACR patient education materials exceeds AMA recommendations, and while ChatGPT significantly lowered the reading level, it failed to reach the 6th-grade level. Key Points • The readability of ACR patient education materials exceeds the AMA reading level recommendations. • AI has the potential to enhance the readability of patient education materials but failed to reach the desired 6th grade reading level. • The inherent bias of ChatGPT must be considered when evaluating the potential for patient education material optimization.