Limits of Generative Pre-Training in Structured EMR Trajectories with Irregular Sampling.

Area of Science:

• Artificial Intelligence
• Biomedical Informatics
• Clinical Data Science

Background:

• Foundation models are increasingly used for clinical prediction with electronic medical records (EMRs).
• Autoregressive pre-training, common in these models, was developed for text generation and its efficacy with irregularly sampled EMR data is unclear.
• Existing research has not sufficiently evaluated the ability of autoregressive models to capture longitudinal clinical context.

Purpose of the Study:

• To evaluate the suitability of autoregressive pre-training for capturing longitudinal clinical context in structured EMRs.
• To examine patient trajectories generated by autoregressive models and assess their fidelity to real-world clinical data.
• To determine if clinically meaningful relationships between variables are preserved over time in generated EMR data.

Main Methods:

• Trained two models: a sequence-to-sequence LSTM and a reduced ETHOS-style Transformer, sharing principles with state-of-the-art EHR foundation models.
• Utilized longitudinal cohorts for HIV antiretroviral therapy (ART) and acute hypotension.
• Evaluated generated trajectories using distributional and correlational fidelity to assess the preservation of variable relationships over time.

Main Results:

• Cross-feature coherence in generated patient trajectories degrades significantly under irregular sampling conditions.
• Autoregressive models may not reliably capture the dynamic patient context present in real-world clinical data.
• The fidelity of generated EMR data to actual patient trajectories is compromised by the nature of EMR sampling.

Conclusions:

• Autoregressive pre-training may not be optimal for capturing longitudinal clinical context in structured EMRs.
• Domain-specific, generation-based evaluation is crucial before applying these models for healthcare transfer learning.
• Findings suggest a need for alternative or adapted pre-training strategies for foundation models in clinical settings.