Automated Extraction of Genetic Eligibility Criteria from Clinical Trial Records Using LLMs - A Technical Case Report

Accurate interpretation of clinical trial eligibility criteria is essential for applications such as patient-trial matching and clinical decision support, particularly in precision oncology. However, relevant information, including genetic mutation requirements, is typically embedded in unstructured text within trial registries such as ClinicalTrials.gov, limiting accessibility for automated processing.

This paper reports on development, integration and evaluation of a system for extracting and structuring mutational eligibility criteria from trial records, focusing on identifying mutated genes, distinguishing inclusion and exclusion criteria, and assigning them to individual study arms. To address challenges like ambiguous abbreviations and context-dependent meaning, we combine large language models (LLMs) for context-aware extraction with rule-based validation against HUGO Gene Nomenclature. The system was implemented using local LLMs and integrated into the Community Annotated Trial Search (CATS) platform.

Applied to 4,918 clinical trials, the system generated structured representations of genetic eligibility criteria for 1,010 studies. Expert review of 42 trials showed that 88.1% of studies were correctly annotated, with a precision of 80% at the level of individual eligibility criteria. Failures were mainly due to hallucinated genes and misinterpreted abbreviations, highlighting challenges in biomedical text processing.

The findings indicate that LLM-based extraction is a promising approach for structuring complex eligibility criteria, particularly when combined with strategies to improve precision. The integration into an operational system demonstrates practical feasibility, while the observed limitations emphasize the need for careful dataset design, error mitigation strategies, and continued refinement to achieve reliable automation in clinical applications.