Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs

Area of Science:

• Medical Informatics
• Natural Language Processing
• Clinical Trial Analysis

Background:

• Randomized Controlled Trials (RCTs) are crucial for determining intervention effectiveness and informing evidence-based care.
• Extracting structured data from unstructured clinical trial reports is a manual, time-consuming process for clinicians.
• Automating the extraction of key trial elements is essential for efficient evidence synthesis.

Purpose of the Study:

• To develop and evaluate a text-to-text model using instruction-tuned Large Language Models (LLMs) for automated extraction of Interventions, Outcomes, and Comparators (ICO elements) from clinical abstracts.
• To infer associated results reported in clinical trial articles.
• To improve the efficiency and accuracy of evidence extraction from scientific literature.

Main Methods:

• Utilized instruction-tuned Large Language Models (LLMs) to build a text-to-text model for joint extraction of ICO elements and associated results.
• Framed evidence extraction as a conditional generation task.
• Conducted manual (expert) and automated evaluations to assess model performance against previous state-of-the-art (SOTA).

Main Results:

• The proposed LLM-based model achieved significant improvements, with approximately a 20-point absolute F1 score increase over the previous SOTA.
• Evaluations demonstrated the effectiveness of framing evidence extraction as a conditional generation task for fine-tuning LLMs.
• Ablation studies and error analyses were performed to understand model performance drivers and identify areas for future enhancement.

Conclusions:

• The developed LLM-based approach substantially enhances the automated extraction of structured evidence from clinical trial abstracts.
• This method offers a more efficient and accurate way to synthesize findings from RCTs, supporting evidence-based medicine.
• A searchable database of structured findings from RCTs up to mid-2022 has been created and released to facilitate access to synthesized trial results.