Enhancing Clinical Trial Selection for Cancer Patients Using Large Language Models

Area of Science:

• Oncology
• Medical Informatics
• Artificial Intelligence

Background:

• Clinical trial matching for cancer patients with specific gene mutations is challenging.
• Existing search tools like ClinicalTrials.gov can yield irrelevant or misleading results.
• Accurate identification of eligible patients is crucial for effective cancer treatment.

Purpose of the Study:

• To evaluate the efficacy of two large language models (LLMs), GPT-4.0 and Gemini 2.0, in assessing patient eligibility for cancer clinical trials.
• To compare the performance of these LLMs against physician-curated benchmarks for specific gene mutations.

Main Methods:

• GPT-4.0 and Gemini 2.0 were prompted with clinical trial data and patient cancer mutation information.
• Model performance was assessed using F1-scores across six gene mutations (ALK, BRAF, EGFR, ERBB2, KIT, KRAS).
• Decision trees were employed to identify key textual indicators used by the LLMs.

Main Results:

• Both LLMs demonstrated good F1-scores, with Gemini 2.0 averaging 70% and GPT-4.0 averaging 64%.
• These results indicate the potential of LLMs to streamline the clinical trial matching process.
• Decision trees offered interpretability by highlighting important textual features influencing LLM evaluations.

Conclusions:

• Proprietary LLMs like GPT-4.0 and Gemini 2.0 can be utilized "off the shelf" for clinical trial eligibility evaluation.
• Limited LLM fine-tuning and patient information are sufficient for this task.
• LLMs offer a feasible approach to improve the accuracy and efficiency of matching cancer patients to relevant clinical trials.