Randomized Controlled Trial Comparing AI-Generated, Clinician-Edited to Human-Generated After-Visit Summaries

To test whether AI-generated, clinician-edited after-visit summaries (AVS) are more patient-friendly than clinician-generated AVS, without compromising safety, when implemented in a live inpatient setting.

Patients discharged from inpatient medicine units between January 22, 2025, and May 23, 2025, were randomized to the intervention arm, which included AI-generated, clinician-edited AVS, or the control arm, which included usual care of clinician-only AVS. Outcomes include text between 6^th and 8^th grade reading levels, and presence of a "simplified" description of both the diagnosis and treatment/interventions. Our composite outcome was positive if all 3 of these outcomes were positive. We also surveyed patients, nurses, and clinicians on perceptions of safety, empathy, and understandability.

Among 61 patients enrolled in the trial, the AI-generated, clinician-edited AVS were significantly more likely to have a mean reading level closer to 6^th to 8^th grade than the control group (8.5 vs 10.6, P <.001). The AI-generated, clinician-edited AVS were also more likely to contain simplified explanations of diagnoses (96.7% vs 12.9%, P <.001) and hospital interventions (86.7% vs 12.9%, P <.001). Our composite outcome favored AI-generated, clinician-edited summaries (13.3% vs 6.5%, p = 0.637).

In this small, randomized trial, exploratory and component measures suggest improved patient-friendliness in AI-generated, clinician-edited AVS, although no significant difference was observed in the primary outcome.

NCT06711458.