A Large-Language Model Framework for Relative Timeline Extraction from PubMed Case Reports.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Clinical Data Analysis

Background:

• Accurate timing of clinical events is crucial for patient trajectory analysis, including forecasting and causal reasoning.
• Electronic health records often lack detailed temporal event data, and clinical reports are unstructured.
• Extracting temporal event information from clinical text is a significant challenge in medical informatics.

Purpose of the Study:

• To develop a system for transforming unstructured clinical case reports into structured textual time series.
• To evaluate the performance of large language models (LLMs) in identifying and localizing clinical events temporally.
• To establish a benchmark for temporal analytics using the PubMed Open Access (PMOA) corpus.

Main Methods:

• A system was developed to create pairs of textual events and timestamps from clinical case reports.
• Manual and LLM annotations were compared on ten randomly sampled PMOA case reports.
• Inter-LLM agreement was assessed on a large dataset of annotations (n=3,103 events from N=93 reports).

Main Results:

• LLM models demonstrated moderate event recall (0.80) in identifying clinical events.
• LLM models achieved high temporal concordance (0.95) among the identified events.
• The study established task, annotation, and assessment systems for temporal analytics.

Conclusions:

• The developed system effectively transforms case reports into time-series data suitable for temporal analytics.
• LLMs show promise for extracting temporally structured clinical event data from unstructured text.
• This work provides a benchmark and methodology for leveraging the PMOA corpus for advanced clinical data analysis.