Automated Extraction of Patient-Centered Outcomes After Breast Cancer Treatment: An Open-Source Large Language...

Area of Science:

• Natural Language Processing in Oncology
• Clinical Informatics
• Patient-Centered Care Research

Background:

• Patient-centered outcomes (PCOs) are crucial for evaluating cancer treatment effectiveness and quality of life.
• Extracting PCOs from unstructured clinical narratives is challenging but vital for comprehensive analysis.
• Existing data on PCOs are often limited in scale and scope.

Purpose of the Study:

• To assess the adaptability of large language models (LLMs) for extracting PCOs from clinical notes.
• To develop and evaluate an open-source framework for fine-tuning LLMs for PCO extraction tasks.
• To compare the performance of different LLMs (GPT-2, BioGPT, PMC-LLaMA) in PCO identification across multiple institutions.

Main Methods:

• Three LLMs (GPT-2, BioGPT, PMC-LLaMA) were evaluated on PCO extraction tasks.
• An open-source framework was developed for fine-tuning LLMs on clinical narratives.
• Models were tested across three institutions: Mayo Clinic, Emory University Hospital, and Stanford University.

Main Results:

• Zero-shot and few-shot LLMs demonstrated poor performance in PCO extraction.
• Fine-tuned, task-specific LLMs significantly outperformed non-fine-tuned models.
• The fine-tuned GPT-2 model showed superior performance compared to larger LLMs like BioGPT and PMC-LLaMA.

Conclusions:

• LLMs require domain-specific fine-tuning for effective application in clinical settings.
• The proposed fine-tuning framework offers an efficient and adaptable approach for PCO information extraction.
• This method has the potential to improve PCO data analysis without extensive computational resources.