Synoptic Reporting by Summarizing Cancer Pathology Reports using Large Language Models.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Artificial Intelligence in Healthcare

Background:

• Synoptic reporting structures clinical information, improving patient care by reducing errors and enhancing report completeness.
• Manual synthesis of synoptic reports from narrative data is labor-intensive and prone to errors, especially with numerous data fields.
• Large Language Models (LLMs) offer advanced natural language processing capabilities with untapped potential in medical applications.

Purpose of the Study:

• To explore the application of state-of-the-art LLMs for the automated generation of synoptic reports.
• To evaluate the effectiveness and challenges of using LLMs in synthesizing structured clinical information.

Main Methods:

• Utilized a dataset of 7,774 narrative pathology reports with corresponding annotated synoptic reports.
• Fine-tuned LLAMA-2, a state-of-the-art LLM, to generate synoptic reports based on 22 unique data elements.
• Evaluated LLM-generated report accuracy using BERT F1 Score and manual validation.

Main Results:

• Fine-tuned LLAMA-2 achieved a BERT F1 Score of 0.86 or higher across all data elements.
• Achieved BERT F1 Scores of 0.94 or higher for over 50% (11 of 22) of the data elements.
• Report accuracies ranged from 76% to 81% for clinical reports.

Conclusions:

• Demonstrated the successful automatic generation of synoptic reports through LLM fine-tuning.
• LLMs show significant potential for improving the efficiency and accuracy of clinical documentation.