Evaluating the Performance of Artificial Intelligence for Improving Readability of Online English- and
Carrie N Reaver1, Daniel E Pereira, Elisa V Carrillo
1Department of Orthopedic Surgery, Washington University in St. Louis, St. Louis, Missouri.
Background:
The readability of most online patient educational materials (OPEMs) in orthopaedic surgery is above the American Medical Association/National Institutes of Health recommended reading level of sixth grade for both English- and Spanish-language content. The current project evaluates ChatGPT's performance across English- and Spanish-language orthopaedic OPEMs when prompted to rewrite the material at a sixth-grade reading level.
Methods:
We performed a cross-sectional study evaluating the readability of 57 English- and 56 Spanish-language publicly available OPEMs found by querying online in both English and Spanish for 6 common orthopaedic procedures. Five distinct, validated readability tests were used to score the OPEMs before and after ChatGPT 4.0 was prompted to rewrite the OPEMs at a sixth-grade reading level. We compared the averages of each readability test, the cumulative average reading grade level, average total word count, average number of complex words (defined as ≥3 syllables), and average number of long sentences (defined as >22 words) between original content and ChatGPT-rewritten content for both languages using paired t tests.
Results:
The cumulative average reading grade level of original English- and Spanish-language OPEMs was 9.6 ± 2.6 and 9.5 ± 1.5, respectively. ChatGPT significantly lowered the reading grade level (improved comprehension) to 7.7 ± 1.9 (95% CI of difference, 1.68 to 2.15; p < 0.05) for English-language content and 8.3 ± 1.3 (95% CI, 1.17 to 1.45; p < 0.05) for Spanish-language content. English-language OPEMs saw a reduction of 2.0 ± 1.8 grade levels, whereas Spanish-language OPEMs saw a reduction of 1.5 ± 1.2 grade levels. Word count, use of complex words, and long sentences were also reduced significantly in both languages while still maintaining high accuracy and similarity compared with original content.
Conclusions:
Our study supports the potential of artificial intelligence as a low-cost, accessible tool to assist health professionals in improving the readability of orthopaedic OPEMs in both English and Spanish.
Related Concept Videos
01:21Health Literacy
01:30Guidelines for Nursing Documentation I
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
02:07Improving Translational Accuracy
01:11Guidelines for Writing Outcome
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
01:24Nursing Process for Patient and Caregiver Teaching I: Assessment and Diagnosis
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
01:13Patient-centered Care