Temporal self-attention for risk prediction from electronic health records using non-stationary kernel approximation

Area of Science:

• Health Informatics
• Machine Learning
• Temporal Data Analysis

Background:

• Electronic Health Records (EHRs) are vital for patient representation, but modeling their inherent non-stationarity remains a challenge.
• Existing methods often assume stationarity, overlooking critical temporal dynamics and domain knowledge embedded in irregular patient visit intervals.
• Disease progression and patient states evolve over time, necessitating models that capture these non-stationary patterns.

Purpose of the Study:

• To introduce a novel method combining self-attention with non-stationary kernel approximation for enhanced EHR patient representation.
• To effectively capture both contextual information and complex temporal relationships within patient visit histories.
• To address the limitations of stationary assumptions in existing EHR modeling techniques.

Main Methods:

• Developed a new approach integrating self-attention mechanisms with non-stationary kernel approximations (e.g., quadratic, cubic, bi-quadratic polynomials).
• Evaluated the method on two large-scale, real-world EHR datasets (general and pregnant patient cohorts) comprising over 76,000 patients.
• Compared the proposed models against baseline (e.g., LSTM, RETAIN) and stationary kernel approximation models using metrics like NDCG@10 and Hit@10.

Main Results:

• Non-stationary kernel models significantly outperformed baseline methods on both NDCG@10 and Hit@10 metrics across both datasets.
• A more substantial performance improvement was observed for the NDCG@10 metric in the general EHR dataset.
• Stationary kernels also showed gains over baselines, performing comparably to non-stationary kernels for Hit@10 in the second dataset.

Conclusions:

• The findings strongly validate the efficacy of non-stationary kernels for temporal modeling of EHR data.
• Accurately modeling non-stationary temporal information is essential for improving healthcare prediction tasks.
• The proposed method offers a more robust approach to patient representation from longitudinal EHR data.