A Large-Language Model Framework for Relative Timeline Extraction from PubMed Case Reports.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Clinical Data Analysis

Background:

• Accurate patient trajectory analysis requires precise timing of clinical events.
• Electronic health records often lack detailed temporal event data.
• Clinical reports are unstructured and lack localized event timestamps.

Purpose of the Study:

• To develop a system for transforming clinical case reports into structured textual time series.
• To evaluate the performance of large language models (LLMs) in extracting temporal event data.
• To establish a benchmark for temporal analytics using the PubMed Open Access (PMOA) corpus.

Main Methods:

• Developed a system to create text-event timestamp pairs from case reports.
• Compared manual annotations with LLM-based annotations on PMOA case reports.
• Assessed inter-LLM agreement for temporal event extraction.

Main Results:

• LLM models demonstrated moderate event recall (0.80).
• LLM models achieved high temporal concordance among identified events (0.95).
• Established task, annotation, and assessment systems for temporal analytics.

Conclusions:

• LLMs can effectively extract temporally structured event data from clinical reports.
• The developed system and findings provide a benchmark for temporal analytics in medicine.
• Leveraging the PMOA corpus with LLMs can enhance patient trajectory analysis.