Large Language Models as Decision-support Tools for Adjuvant Therapy Planning in Early-stage Hormone...

Area of Science:

• Oncology
• Artificial Intelligence
• Clinical Decision Support

Background:

• Adjuvant treatment decisions for hormone receptor-positive (HR), HER2-negative early-stage breast cancer often rely on multigene assays.
• Limited access to genomic testing poses a challenge, particularly in resource-limited settings.

Purpose of the Study:

• To evaluate the concordance of large language models (LLMs) with medical oncologist recommendations for adjuvant therapy.
• To assess LLM performance in HR+/HER2- early-stage breast cancer patients without genomic assay results.

Main Methods:

• Clinical and pathological data from 411 patients were input into ChatGPT-4o and ChatGPT-o3.
• LLM recommendations (chemotherapy + endocrine therapy vs. endocrine therapy alone) were compared to an oncologist's.
• Agreement was statistically analyzed using Fleiss's and Cohen's kappa statistics.

Main Results:

• Overall agreement among the clinician and both LLMs was substantial (κ=0.67).
• ChatGPT-4o showed moderate agreement with the clinician (κ=0.60), slightly higher than ChatGPT-o3 (κ=0.55).
• Agreement between the two LLMs was almost perfect (κ=0.88), with ChatGPT-4o aligning more closely with clinical judgment.

Conclusions:

• LLMs demonstrate substantial concordance with clinician decision-making for adjuvant therapy in HR+/HER2- early-stage breast cancer without genomic data.
• These models can function as supportive tools, aiding oncologists, especially where multigene assays are inaccessible.