A Masked Language Model for Multi-Source EHR Trajectories Contextual Representation Learning.

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Clinical Decision Support Systems

Background:

• Leveraging electronic health records (EHR) with machine learning (ML) is crucial for advancing clinical decision-making.
• Existing ML models face challenges in capturing long/short-term dependencies and complex interactions between diseases and interventions.
• Bidirectional transformers have shown success in addressing temporal dependencies in EHR data.

Purpose of the Study:

• To address the challenge of modeling interactions between diseases and interventions using EHR data.
• To develop a novel ML approach for predicting one data source (e.g., disease codes) from others (e.g., intervention codes).
• To enhance the predictive power of ML models for clinical decision support.

Main Methods:

• Utilized a transformer-based architecture, a type of neural network adept at handling sequential data.
• Employed a masking strategy, where one data source (International Classification of Diseases, Tenth Revision [ICD10] codes) was hidden.
• Trained the transformer to predict the masked ICD10 codes using other available data sources, such as the Anatomical Therapeutic Chemical (ATC) classification system codes.

Main Results:

• The proposed method effectively models the intricate relationships between diseases and their associated interventions.
• Successfully predicted masked disease codes (ICD10) by leveraging information from other data modalities (ATC codes).
• Demonstrated the capability of transformers to capture cross-modal interactions within EHR data.

Conclusions:

• This novel transformer-based approach can effectively capture and model disease-intervention interactions in EHR data.
• The masking and prediction strategy offers a viable method for integrating diverse data sources in clinical ML.
• This research advances the potential of ML for more accurate and comprehensive clinical decision support systems.