Evaluating the use of advanced large language models to improve readability of head and neck cancer patient education materials

To evaluate whether advanced large language models (LLMs) ChatGPT-4o, ChatGPT o3, Microsoft Copilot, Claude Sonnet 4, Gemini 1.5 Flash, and DeepSeek V3-R1 can improve the readability of head and neck cancer patient education materials while maintaining accuracy. Eleven publicly available articles were assessed for baseline readability using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease Score (FRES). Each article was rewritten by six LLMs using a standardized prompt to simplify text to a sixth-grade level. Readability was reassessed post-intervention, and paired t-tests with repeated-measures ANOVA and Bonferroni correction compared performance across models. Baseline readability was poor, with a mean FKGL of 10.1 and FRES of 51.5, exceeding national recommendations. Gemini, Copilot, ChatGPT o3, and DeepSeek significantly reduced FKGL compared to originals, while ChatGPT-4o showed minimal change and Claude increased difficulty. Gemini performed best, achieving FKGL ≤7 in 81.8 % of cases, followed by Copilot and DeepSeek at 72.7 %. ChatGPT-4o met this threshold in 18.2 % of articles, and Claude met it in none. LLMs can improve the readability of head and neck cancer patient education materials, but effectiveness varies substantially among models. Gemini, Copilot, and DeepSeek were most successful in meeting clinical readability thresholds, whereas ChatGPT-4o and Claude underperformed. Careful model selection and clinical oversight are essential when applying AI to patient education, as rewritten materials must still be reviewed for accuracy before public use.