Large language models are powerful electronic health record encoders

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Prediction Modeling

Background:

• Electronic health records (EHRs) hold significant potential for clinical prediction.
• The complexity and heterogeneity of EHR data challenge traditional machine learning approaches.
• Domain-specific EHR foundation models improve accuracy but face data access and vocabulary limitations.

Purpose of the Study:

• To develop a method for utilizing general-purpose large language models (LLMs) for clinical prediction using EHR data.
• To overcome limitations of data access and site-specific vocabularies in EHR foundation model development.
• To evaluate the performance of LLM-based embeddings against specialized EHR models.

Main Methods:

• EHR data was converted into plain text by replacing medical codes with natural-language descriptions.
• General-purpose LLMs were used to generate high-dimensional embeddings from the text-based EHR data.
• LLM-based embeddings were evaluated on 15 clinical tasks from the EHRSHOT benchmark and external validation using UK Biobank data.

Main Results:

• LLM-based embeddings performed comparably to a specialized EHR foundation model (CLMBR-T-Base) across 15 clinical tasks.
• External validation demonstrated statistically significant improvements for some tasks using the LLM-based model.
• Improvements were attributed to higher vocabulary coverage and better generalization of the LLM-based approach.

Conclusions:

• Converting EHRs to text enables general-purpose LLMs to create effective clinical prediction embeddings without private data access.
• LLM-based embeddings offer portability and data independence, contrasting with the computational efficiency of specialized EHR models.
• A trade-off exists between the efficiency of specialized models and the flexibility of LLM-based approaches for EHR data analysis.