A Natural Language Processing Framework for Structuring and Visualizing Clinical Trial Eligibility Criteria at Scale:...

Area of Science:

• Computational biology
• Bioinformatics
• Clinical informatics

Background:

• Clinical trial eligibility criteria are crucial but often complex, hindering patient recruitment and data analysis.
• Manual review of criteria is inefficient and prone to errors.
• Natural language processing (NLP) and large language models (LLMs) offer potential solutions for standardizing and analyzing eligibility criteria at scale.

Purpose of the Study:

• To develop and evaluate a scalable system using LLM-enabled NLP and unsupervised learning to identify, normalize, categorize, and visualize clinical trial eligibility criteria.
• To enhance patient-trial matching and identify domain-level trends in trial eligibility.

Main Methods:

• A three-part pipeline was designed: text representation with embeddings and clustering, dual-layer LLM summarization for normalization and deduplication, and an interactive visualization interface.
• The pipeline was applied to 53,872 oncology trials from ClinicalTrials.gov.
• Feasibility was assessed through end-to-end processing and validation of cluster coherence and domain-specific patterns.

Main Results:

• The system successfully processed all trials, generating stable clusters of inclusion/exclusion criteria.
• LLM summarization produced concise, non-redundant labels, improving interpretability.
• The visualization interface facilitated exploration of cross-trial patterns and temporal trends, identifying enrollment barriers.
• Human validation confirmed 94% accuracy in categorizing criteria.

Conclusions:

• A combined embeddings-clustering-LLM pipeline effectively standardizes heterogeneous eligibility text and reveals domain-level patterns.
• This framework accelerates patient-trial matching and informs future clinical trial design.
• The approach is generalizable to other diseases and adaptable to different modeling configurations.