Related Experiment Video
Updated: Mar 7, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Matching Patients With Cell Surface-Targeted Clinical Trials Using Large Language Models
S Carson Callahan1, Matthew R Chrostek1, Nicholas Rydzewski1,2
1Department of Human Oncology, University of Wisconsin, Madison, WI.
Purpose:
Cell surface-targeted therapies (CSTs) are a rapidly expanding class of cancer treatments with high specificity and reduced toxicity. Matching patients who express specific targets to CST clinical trials remains challenging because of complex eligibility criteria, diverse targets, and the absence of centralized, up-to-date trial databases. These gaps limit patient access and contribute to poor trial accrual.
Methods:
We developed a large language model (LLM)-driven pipeline to identify and annotate CST clinical trials. Using a two-pronged approach, LLMs extracted target information from ClinicalTrials.gov and the National Cancer Institute Drug Database. Eight LLMs, including GPT-4o and several open-source models, were benchmarked against manually curated data sets of 814 CST trials and 814 non-CST trials. We evaluated model performance at target and trial levels and analyzed sources of error. We also provide an up-to-date database of open CST trials and their targets from the >100,000 total oncology clinical trials in ClinicalTrials.gov.
Results:
GPT-4o achieved the highest accuracy in identifying CST trials (96.5%) and their targets (89.5%). Combining data sources improved performance, and accuracy increased with later trial phases. Most errors stemmed from vague therapy descriptions or string-matching issues. The model matched 94% of US trials and >95% of trials globally, with exceptions in China and New Zealand. In predicting cell surface localization, Gemma 3:27b and MedLlama3 correctly labeled all known clinical cell surface targets although performance varied beyond the most well-known CSTs.
Conclusion:
Our LLM-based approach enables real-time, automated matching of patients to CST clinical trials, addressing major barriers to enrollment and expanding trial accessibility. Errors were uncommon, and performance is poised to improve as LLMs evolve. Optimizing patient-trial matching for CSTs can improve both patient benefit and trial success.
Insights
A new large language model (LLM) pipeline accurately identifies cell surface-targeted therapy (CST) clinical trials. This technology improves patient access to novel cancer treatments by automating trial matching.
Area of Science:
- Oncology
- Biotechnology
- Artificial Intelligence
Background:
- Cell surface-targeted therapies (CSTs) offer precise cancer treatment with reduced toxicity.
- Patient matching to CST clinical trials is hindered by complex criteria and fragmented databases, limiting access and accrual.
Purpose of the Study:
- To develop and evaluate a large language model (LLM)-driven pipeline for automated identification and annotation of CST clinical trials.
- To create an up-to-date database of open CST trials and their targets.
Main Methods:
- Utilized a two-pronged LLM approach to extract target information from ClinicalTrials.gov and the National Cancer Institute Drug Database.
- Benchmarked eight LLMs, including GPT-4o, against manually curated data of 814 CST and 814 non-CST trials.
- Evaluated model performance at target and trial levels, analyzing error sources.
Main Results:
- GPT-4o demonstrated high accuracy in identifying CST trials (96.5%) and targets (89.5%).
- Performance improved with combined data sources and later trial phases; errors were mainly due to vague descriptions.
- The model successfully matched a high percentage of global trials, with Gemma 3:27b and MedLlama3 showing promise in cell surface target prediction.
Conclusions:
- The LLM-based approach facilitates real-time, automated patient-trial matching for CSTs, enhancing accessibility.
- This method addresses key barriers to clinical trial enrollment and success.
- Continued LLM evolution is expected to further optimize patient-trial matching for improved outcomes.
