Decoding Clinician Authorial Style: A Style-Informed Pipeline for Clinical Document Summary Generation with Large...

Area of Science:

• Artificial Intelligence
• Clinical Informatics
• Natural Language Processing

Background:

• Large language models (LLMs) automate clinical document summarization.
• Current LLM outputs lack individual clinician writing styles, requiring extensive post-editing.
• A significant stylistic gap exists between AI-generated and clinician-authored summaries.

Purpose of the Study:

• To address the stylistic gap in LLM-generated clinical summaries.
• To develop a style-informed generation framework for personalized clinical documentation.
• To evaluate the effectiveness of style-informed generation in matching clinician writing styles.

Main Methods:

• Utilized a multi-author corpus of de-identified clinical summaries.
• Developed a framework extracting clinician-specific stylistic features via LLM feedback.
• Employed a Train→Generate paradigm for personalized summary production.
• Evaluated metrics including ROUGE, BERTScore, cosine similarity, Jaro-Winkler, and BLEU.
• Conducted blinded A/B testing with clinicians comparing AI-generated and clinician-authored summaries.

Main Results:

• Conventional metrics showed limited ability to differentiate writing styles.
• LLM-guided feature extraction improved authorship classification accuracy to 73%.
• Gemini 2.5 Pro pipeline drafts were preferred at rates comparable to or exceeding clinician-authored summaries.
• GPT-4 drafts were preferred less often than original notes.
• Hallucination risks were mitigated through prompt engineering and source-only data constraints.

Conclusions:

• Style-informed generation significantly reduces the stylistic gap in clinical summaries.
• The proposed framework produces clinically acceptable summaries that align with clinician voice.
• Personalized AI-generated summaries show potential to improve efficiency and user satisfaction in clinical documentation.