Using large language models for safety-related table summarization in clinical study reports.

Area of Science:

• Clinical Documentation Automation
• Artificial Intelligence in Healthcare
• Natural Language Processing

Background:

• Clinical trial documentation, particularly safety table summarization in Clinical Study Reports (CSRs), presents a complex challenge.
• Large Language Models (LLMs) offer a potential solution for automating the generation of these structured documents.
• A competitive challenge was organized to explore LLM capabilities in this area.

Observation:

• Multiple teams utilized prompt engineering with Generative Pre-trained Transformer (GPT) models to generate safety table summaries.
• An evaluation framework combining automated metrics and expert reviews assessed the quality of AI-generated documents.
• Performance varied across different LLM solutions, with notable differences in factual accuracy and conciseness.

Findings:

• LLMs demonstrate significant potential for automating the summarization of safety tables within CSRs.
• Factual accuracy and lean writing were identified as key areas where LLM performance varied.
• Prompt engineering with GPT models was a common approach among participating teams.

Implications:

• The results highlight the feasibility of using LLMs for clinical trial documentation automation.
• Areas for improvement include enhancing table data ingestion, incorporating contextual information, and model fine-tuning.
• Continued research and human involvement are essential for maximizing the benefits of LLMs in clinical reporting.