Retrieval-enhanced drafting of ClinicalTrials.gov data elements from clinical protocols

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Biomedical Data Management

Background:

• Manual submission of clinical trial data to ClinicalTrials.gov is time-consuming and prone to errors.
• Inconsistencies in registry entries affect data completeness and reliability.
• Large language models (LLMs) show potential for automating and improving this process.

Purpose of the Study:

• To develop and evaluate ChatCT, a pilot retrieval-augmented LLM system, for drafting ClinicalTrials.gov registry elements.
• To assess the system's ability to generate accurate and compliant registry entries.

Main Methods:

• ChatCT was developed as a retrieval-augmented system to draft registry elements.
• Evaluation focused on semantic similarity, formatting compliance, and concept coverage.
• Metrics included BERTScore F1, field completeness, and ontology-based precision.

Main Results:

• ChatCT drafts demonstrated high semantic similarity to human-authored records (BERTScore F1 ≈ 0.82).
• Formatting compliance was strong for structured data (e.g., Study Design, Arms/Interventions) but varied for narrative sections.
• Ontology-based concept extraction achieved high precision (90%-100%).

Conclusions:

• Retrieval-augmented LLMs can generate drafts for ClinicalTrials.gov that retain key protocol details and meet most formatting needs.
• Post-processing is still required for full submission readiness.
• ChatCT-assisted drafting shows promise for enhancing consistency between protocols and public trial data.