Structured Clinical Input-Guided Large Language Model Workflow for Acute Ischemic Stroke Discharge Education: A...

Large language models (LLMs) may support patient education, but their clinical use remains challenging. We aimed to evaluate the preliminary feasibility of a structured clinical input-guided LLM workflow for generating discharge education drafts for patients with acute ischemic stroke.

Patients with acute ischemic stroke discharged from 6 tertiary stroke centers in China between September 1, 2024, and March 31, 2025, were included. The workflow comprised electronic medical record-based data extraction, mapping to a predefined deidentified case report form, manual verification, standardized prompt-based LLM draft generation, and clinician-facing draft output. For each patient, Chinese-language discharge education drafts were generated using GPT-4o, Grok-3, and DeepSeek-R1. Physician-written discharge instructions prepared during routine clinical practice served as reference materials. Two blinded senior neurologists evaluated the materials across 5 predefined domains. Patient-centered evaluation was conducted in 50 patients. Interrater agreement between the 2 neurologists was assessed using intraclass correlation coefficients. Group comparisons were performed using Friedman tests followed by Bonferroni-corrected Wilcoxon signed-rank tests.

A total of 67 patients with acute ischemic stroke were included. The mean age was 63.6±11.0 years, and 45 patients were men (67.2%). Interrater agreement was good to excellent, with intraclass correlation coefficients ranging from 0.862 to 0.943. Post hoc analyses showed that each LLM-generated draft group received higher expert ratings than physician-written discharge instructions in risk factor control, rehabilitation guidance, follow-up planning, and health education (all Bonferroni-adjusted P<0.001), whereas no pairwise difference in medication management remained significant after correction. Patient-centered ratings were generally favorable, and GPT-4o and Grok-3 received higher empathy ratings than physician-written discharge instructions (both Bonferroni-adjusted P<0.01). No clear hallucinations were identified during expert review, and LLM drafts had fewer unacceptable ratings.

A structured clinical input-guided LLM workflow showed preliminary feasibility for generating clinician-supervised acute ischemic stroke discharge education drafts. Prospective implementation studies are warranted to evaluate workflow integration, usability, and effects on patient-reported and clinical outcomes.