A context-augmented large language model for accurate precision oncology medicine recommendations

Area of Science:

• Oncology
• Medical Informatics
• Artificial Intelligence

Background:

• Precision oncology relies on molecularly informed therapies and evolving regulatory approvals.
• Current large language models (LLMs) lack the specialized knowledge for up-to-date, niche treatment recommendations in oncology.
• Integrating advanced therapies into patient care presents challenges for oncologists.

Purpose of the Study:

• To develop and evaluate a retrieval-augmented generation (RAG)-LLM workflow for precision oncology.
• To benchmark the RAG-LLM approach against a standard LLM for biomarker-driven treatment recommendations.
• To explore strategies for enhancing LLM performance in clinical oncology settings.

Main Methods:

• Developed a RAG-LLM workflow incorporating the Molecular Oncology Almanac (MOAlmanac).
• Benchmarked the RAG-LLM against an LLM-only approach using synthetic and real-world clinical queries.
• Investigated various prompting and retrieval strategies to optimize performance.

Main Results:

• The RAG-LLM achieved up to 95% accuracy on synthetic queries.
• The RAG-LLM demonstrated 93% accuracy on real-world queries from practicing oncologists.
• Identified effective prompting and retrieval strategies for improved LLM performance.

Conclusions:

• The developed RAG-LLM workflow shows significant potential for supporting precision oncology.
• This approach can provide accurate, up-to-date treatment recommendations for cancer patients.
• The study offers guidance for deploying LLMs in clinical settings to aid cancer treatment decisions.