Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction.

Area of Science:

• Artificial Intelligence
• Clinical Informatics
• Natural Language Processing

Background:

• Large language models (LLMs) demonstrate potential for clinical text summarization.
• Their efficacy with long patient trajectories and multi-modal data over time is underexplored.

Purpose of the Study:

• To systematically evaluate open-source LLMs, Retrieval Augmented Generation (RAG), and chain-of-thought (CoT) prompting for long-context clinical summarization and prediction.
• To assess LLM capabilities in synthesizing structured and unstructured Electronic Health Records (EHR) data while maintaining temporal coherence.

Main Methods:

• Re-engineered existing tasks: discharge summarization and diagnosis prediction.
• Utilized two publicly available EHR datasets.
• Evaluated state-of-the-art open-source LLMs with RAG and CoT prompting.

Main Results:

• Long context windows improve data integration but not consistently clinical reasoning.
• LLMs exhibit limitations in handling temporal progression and predicting rare diseases.
• RAG demonstrated some reduction in hallucinations but did not fully overcome existing limitations.

Conclusions:

• This study addresses a gap in long clinical text summarization and temporal reasoning with LLMs.
• Establishes a foundation for evaluating LLMs on multi-modal clinical data and temporal coherence.
• Highlights ongoing challenges for LLMs in complex clinical data analysis.