PRISM: Patient Records Interpretation for Semantic clinical trial Matching system using large language models

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Research

Background:

• Clinical trial matching is crucial but labor-intensive, often relying on manual review of electronic health records (EHRs).
• Current automated methods using Large Language Models (LLMs) are limited by synthetic datasets, failing to capture real-world data complexities.
• Patients often miss potential therapeutic options due to inefficiencies in the matching process.

Purpose of the Study:

• To conduct a large-scale empirical evaluation of an automated clinical trial matching system using real-world EHR data.
• To compare the performance of a custom-tuned LLM, OncoLLM, against proprietary LLMs and human experts.
• To demonstrate the feasibility and effectiveness of LLMs in real-world clinical trial eligibility screening.

Main Methods:

• Developed and fine-tuned a custom LLM, OncoLLM, specifically for clinical trial matching.
• Conducted comprehensive experiments using a large dataset of real-world electronic health records (EHRs).
• Evaluated OncoLLM's performance against GPT-3.5 and a panel of qualified medical doctors.

Main Results:

• OncoLLM demonstrated superior performance compared to GPT-3.5 in clinical trial matching.
• The custom LLM achieved performance comparable to that of qualified medical doctors.
• The study validates the use of LLMs with real-world EHRs for efficient patient-trial matching.

Conclusions:

• Automated clinical trial matching using LLMs, particularly custom-tuned models like OncoLLM, is effective with real-world data.
• OncoLLM offers a promising solution to improve patient access to clinical trials by overcoming limitations of current methods.
• This approach can significantly reduce the manual effort and time required for clinical trial eligibility assessment.