Enhancing hepatopathy clinical trial efficiency: a secure, large language model-powered pre-screening pipeline

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Trial Recruitment
• Health Informatics

Background:

• Patient recruitment for liver disease cohorts (e.g., hepatocellular carcinoma, liver cirrhosis) involves complex criteria interpretation.
• Manual screening is time-consuming and error-prone.
• Existing AI pre-screening methods face challenges in accuracy, efficiency, and data privacy.

Purpose of the Study:

• To develop a novel, precise, safe, and efficient AI pipeline for patient pre-screening in clinical trials.
• To leverage large language models (LLMs) guided by clinical expertise.
• To address challenges in semantic criteria interpretation for complex liver diseases.

Main Methods:

• Developed a pipeline breaking down complex criteria into composite questions.
• Employed two LLM strategies for semantic question-answering in electronic health records: (1) Anthropomorphized Experts' Chain of Thought (Pathway A), and (2) Preset Stances within Agent Collaboration (Pathway B).
• Evaluated pipeline performance using precision, recall, time consumption, and counterfactual inference.

Main Results:

• Achieved high precision (0.921 at criteria level) and good recall (~0.82 at criteria level) with high efficiency (0.44s per task).
• Pathway B demonstrated high precision in complex reasoning; Pathway A excelled in direct data extraction with robust precision and recall.
• Pipeline showed promising precision in hepatocellular carcinoma (0.878) and cirrhosis (0.843) trials.

Conclusions:

• The developed pipeline is data-secure, time-efficient, and achieves high precision and good recall in hepatopathy trials.
• Offers a promising solution for streamlining clinical trial workflows and improving patient recruitment.
• Its efficiency, adaptability, and performance make it suitable for resource-constrained clinical settings.