TEE4EHR: Transformer event encoder for better representation learning in electronic health records

Area of Science:

• Machine Learning
• Biomedical Informatics
• Time Series Analysis

Background:

• Electronic health records (EHRs) present challenges for machine learning due to irregular data sampling and non-random missing values.
• Developing accurate predictive models requires addressing these data complexities inherent in clinical data.

Purpose of the Study:

• To introduce TEE4EHR, a transformer event encoder (TEE) model incorporating point process loss for analyzing event sequences in EHRs.
• To evaluate TEE4EHR's effectiveness in both self-supervised learning and downstream clinical prediction tasks using benchmark and real-world EHR datasets.

Main Methods:

• Developed TEE4EHR, a transformer event encoder (TEE) with point process loss to encode laboratory test patterns in EHRs.
• Employed a self-supervised learning approach, jointly training the TEE with an attention-based deep neural network.
• Proposed an algorithm for aggregating attention weights to understand event interactions.
• Transferred and froze the learned TEE for downstream outcome prediction tasks.

Main Results:

• The TEE demonstrated superior performance in negative log-likelihood and future event prediction within a self-supervised learning framework.
• TEE4EHR outperformed state-of-the-art models in handling irregularly sampled time series for clinical outcome prediction.
• The model effectively improved representation learning in EHRs, showing utility for clinical prediction.

Conclusions:

• TEE4EHR offers a robust solution for machine learning on irregularly sampled EHR data.
• The model enhances predictive accuracy for clinical outcomes and improves the interpretability of event interactions.
• This approach advances the application of deep learning in EHR analysis and clinical decision support.