MedRep: medical concept representations for general electronic health record foundation models

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Biomedical Data Science

Background:

• Electronic health record (EHR) foundation models struggle with generalizability due to diverse medical codes across institutions.
• Lack of standardized medical concept representation hinders model performance and interoperability.

Purpose of the Study:

• Introduce Medical Concept Representation (MedRep) to standardize EHR foundation model inputs.
• Enable models to recognize semantically similar concepts irrespective of specific medical code IDs.
• Enhance the generalizability and cooperative potential of EHR foundation models.

Main Methods:

• Utilized Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) vocabulary (7.5 million concepts, 66 vocabularies).
• Integrated large language model (LLM)-generated concept descriptions and OMOP graph ontology via graph contrastive learning and knowledge distillation.
• Evaluated MedRep on MIMIC-IV (internal) and EHRSHOT (external) datasets across 9 prediction tasks.

Main Results:

• MedRep significantly outperformed baseline models, especially in external validation (avg. 0.088 AUC-ROC, 0.208 AUC-PR improvements).
• MedRep-based models identified more clinically relevant concepts, enhancing interpretability.
• Performance gains were consistent across various EHR foundation model architectures (BEHRT, Med-BERT, CDM-BERT).

Conclusions:

• MedRep improves EHR foundation model generalizability by creating similar representations for similar concepts.
• Facilitates knowledge merging from multiple hospital datasets and reduces healthcare disparities.
• Establishes MedRep as a standard baseline representation for OMOP CDM-adopting EHR foundation models.