A Comparative Cross-sectional Study Assessing the Readability of ChatGPT and UpToDate Content on Multiple Endocrine...

With growing reliance on digital health tools, the readability of online medical content is increasingly vital in patient-centered care. Artificial intelligence platforms like ChatGPT are widely used to access medical information, yet their suitability for conveying complex conditions such as multiple endocrine neoplasia (MEN) syndromes remains underexplored.

This study aimed to compare the readability of MEN-related educational materials generated by ChatGPT with those from the evidence-based platform UpToDate (UTD), evaluating their appropriateness for patient education.

Six related subjects that addressed MEN types 1 and 2 were chosen. The WebFX readability tool was used to examine texts produced by ChatGPT and the related UTD articles. Word count, sentence count, proportion of difficult words, Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), and Simple Measure of Gobbledygook Index were among the metrics. R (v4.3.2) and SPSS (v25) were used for statistical analysis, and the Wilcoxon signed-rank test was used.

ChatGPT responses were significantly shorter (median word count: 594.0 vs. 1955.5) and contained fewer sentences (66.0 vs. 166.0), though sentence complexity differences were not significant. Both sources scored poorly in readability: ChatGPT had an FRE of 21.3 and FKGL of 15.3, while UTD scored 24.9 and 16.7, respectively. Although ChatGPT used fewer difficult words, it had a higher proportion of complex terms (35.6% vs. 30.1%; P = 0.0306).

Although both sources went over the suggested readability limits for patient materials, ChatGPT produced more condensed text. To improve understanding and promote health fairness, medical terminology must be made simpler.