LLM-powered breast cancer staging from PET/CT reports: a comparative performance study.

Area of Science:

• Artificial Intelligence in Oncology
• Natural Language Processing (NLP) for Medical Reporting
• Breast Cancer Staging and Classification

Background:

• Accurate tumor-node-metastasis (TNM) classification is vital for breast cancer management, influencing treatment and prognosis.
• Current staging relies on interpreting imaging reports, which can be time-consuming and prone to variability.
• Large language models (LLMs) present an opportunity to automate and standardize TNM staging extraction from clinical reports.

Purpose of the Study:

• To evaluate the performance of four LLMs (ChatGPT-4o, DeepSeek V3, Claude 3.5 Sonnet, Gemini 2.0 Flash) in extracting TNM staging from PET/CT reports.
• To assess the accuracy of LLM-derived TNM classifications and UICC stages against expert-generated benchmarks.
• To explore the potential of LLMs in enhancing the accuracy and efficiency of oncologic workflows in breast cancer care.

Main Methods:

• A dataset of 111 fictitious PET/CT reports for breast cancer was curated.
• Four advanced LLMs were tasked with determining TNM staging and UICC stage based on established guidelines.
• Model outputs were systematically compared against expert-defined TNM and UICC stage classifications.

Main Results:

• Claude 3.5 Sonnet achieved superior performance across all metrics, demonstrating high F1 scores for T, N, and M classifications.
• Claude 3.5 Sonnet obtained an F1 score of 0.92% for UICC stage classification.
• The study indicates strong potential for LLMs to accurately perform TNM staging from imaging reports.

Conclusions:

• Advanced NLP, specifically LLMs like Claude 3.5 Sonnet, can reliably support cancer staging processes.
• LLM-based systems show promise in improving the accuracy of oncologic workflows and aiding clinical decision-making.
• Further validation through prospective clinical trials and diverse settings is necessary to confirm these findings and facilitate AI adoption in breast cancer management.