Synoptic reporting by summarizing cancer pathology reports using large language models.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Pathology

Background:

• Synoptic reporting enhances patient care through structured clinical information but creates administrative burdens for physicians.
• The application of Large Language Models (LLMs) for automating synoptic reporting is an underexplored area.

Purpose of the Study:

• To explore the potential of state-of-the-art LLMs for automating synoptic reporting in pathology.
• To develop and validate a comprehensive automation framework for synoptic reporting using LLMs.

Main Methods:

• Utilized a dataset of 7774 pathology reports across 8 cancer types, paired with physician-annotated synoptic reports.
• Developed an automation framework integrating LLMs with parameter-efficient optimization, scalable prompt templates, and robust evaluation strategies.
• Validated the framework on internal and external data, ensuring alignment with pathologist responses.

Main Results:

• Fine-tuned LLAMA-2 achieved BERT F1 scores above 0.86 across all data elements.
• LLAMA-2 exceeded 0.94 F1 scores on over 50% (11 of 22) of data elements.
• Manually assessed mean semantic accuracies reached 77%, with up to 81% for short clinical reports.

Conclusions:

• The developed LLM framework demonstrates significant potential for automating synoptic reporting in pathology.
• Automated synoptic reporting can improve efficiency and accuracy, potentially alleviating physician administrative burdens.
• This approach shows promise for enhancing the quality and completeness of pathology reports.