Comparative Analysis of Japanese Clinical Note Styles Between Physicians and Large Language Models Using Identical...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Natural Language Processing
• Psychiatric Informatics

Background:

• Large language models (LLMs) are increasingly used for clinical documentation.
• The ability of LLMs to replicate specialty-specific writing styles in psychiatry is not well understood.

Purpose of the Study:

• To systematically compare the narrative styles of human physicians and LLMs in psychiatric case documentation.
• To evaluate LLM replication of specialty-specific documentation patterns.

Main Methods:

• Compared 134 human-authored and LLM-generated psychiatric notes (major depressive disorder, schizophrenia) using lexical, syntactic, semantic, and topic-level analyses.
• Utilized metrics such as BLEU, ROUGE-L, BERTScore, and TER, alongside redundancy and medical term variation analyses.

Main Results:

• LLM notes were longer, more repetitive, and less lexically diverse than human notes.
• LLMs exhibited a uniform, template-like style, contrasting with physicians' flexible, context-sensitive approach.
• LLM notes used more abstract language and showed less variation in clinical information emphasis.

Conclusions:

• LLMs can imitate surface-level stylistic features but fail to capture nuanced, diagnostically relevant content.
• Future clinical applications require optimized prompt design or fine-tuning for narrative depth and clinical relevance.