Multidimensional linguistic evaluation of LLM-generated and resident-rewritten discharge summaries
Related Experiment Video
Updated: Nov 8, 2026
Fine-Tuning Large Language Models Using Entity Hallucination Index for Text Summarization
Published on: January 9, 2026
Raphaell Pinto de Arruda Trindade1, Lucas Pla Cid Senra2, Gustavo Freitas Feitosa2
1Faculdade Israelita de Ciências da Saúde Albert Einstein, São Paulo, Brazil; Hospital Santa Izabel, Santa Casa da Bahia, Salvador, Bahia, Brazil.
Background:
Clinical documentation frequently involves complex terminology and dense information that challenges plain language communication during care transitions. Large Language Models (LLM) have increasingly been used to generate easier-to-understand medical texts, although multidimensional linguistic evaluations of their outputs remain limited. This study compared GPT-5-generated and resident-rewritten versions of hospital discharge summaries in producing patient-friendly language.
Methods:
Discharge summaries were randomly allocated to medical residents in training programs for rewriting and were also rewritten by GPT-5, an OpenAI model. These summaries were evaluated for readability and textual comprehensibility using quantitative linguistic metrics across lexical, semantic, cohesion, and discourse domains. Prompt instructions were standardized through prompt engineering procedures. Pairwise comparisons were performed via the Wilcoxon signed-rank test, and Spearman correlation coefficients were used to examine associations across linguistic variables.
Results:
Fifty-nine paired discharge summaries were analyzed. LLM-generated texts demonstrated greater readability and lexical simplification, stronger referential cohesion and greater textual predictability with more familiar and imageable vocabulary. Conversely, resident-rewritten discharge summaries presented greater informational density with lower semantic ambiguity and higher lexical diversity. Overall syntactic complexity remained similar between groups. Correlation analyses suggested a more clustered, domain-segregated pattern of associations in LLM outputs, whereas resident-rewritten summaries showed associations spanning a broader range of linguistic domains.
Conclusions:
The evaluated LLM improved discharge-summary readability and accessibility, whereas resident-rewritten summaries preserved greater semantic specificity and informational density. Human oversight and validation with patients remain essential to ensure clinically reliable simplification of patient-facing discharge documents.
