Expert-Guided Large Language Models for Clinical Decision Support in Precision Oncology

Area of Science:

• Oncology
• Medical Informatics
• Artificial Intelligence

Background:

• The rapid growth of medical literature presents challenges for oncologists seeking targeted therapies.
• General-purpose large language models (LLMs) have limited clinical utility due to a lack of domain-specific knowledge.

Purpose of the Study:

• To introduce MEREDITH, an LLM system designed to support treatment recommendations in precision oncology.
• To leverage Google's Gemini Pro LLM with retrieval-augmented generation and chain-of-thought methodologies.

Main Methods:

• MEREDITH was evaluated on 10 fictional oncology cases with iterative feedback from a molecular tumor board (MTB).
• The system was enhanced to include clinical studies, trial databases, drug approval status, and oncologic guidelines.
• Qualitative and quantitative assessments were performed, including measuring semantic cosine similarity between LLM suggestions and clinician responses.

Main Results:

• MEREDITH identified a median of 4 treatment options, compared to 2 by MTB experts, including therapies based on preclinical data and combination treatments.
• Incorporating a curated medical dataset contextualizing molecular targetability broadened treatment possibilities.
• High concordance (94.7%) between MEREDITH and expert recommendations was achieved, with a significant increase in semantic similarity (0.71 to 0.76, P = .01).

Conclusions:

• Expert feedback and domain-specific data are crucial for augmenting LLM performance in oncology.
• Further research is needed to explore the responsible integration of LLMs into clinical workflows.