Related Experiment Video
Updated: Aug 8, 2025
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records
Emmi Antikainen1, Joonas Linnosmaa2, Adil Umer2
1VTT Technical Research Centre of Finland Ltd., 33101, Tampere, Finland. emmi.antikainen@gmail.com.
Insights
Deep learning models predict cardiovascular disease (CVD) patient mortality risk using electronic health records (EHR). XLNet shows improved recall over BERT for six-month mortality prediction in CVD patients.
Area of Science:
- Artificial Intelligence in Medicine
- Cardiovascular Disease Research
- Health Informatics
Background:
- Cardiovascular diseases (CVDs) are a leading cause of death globally, causing significant morbidity and healthcare costs.
- Accurate prediction of mortality risk in CVD patients is crucial for timely intervention and improved patient outcomes.
- Electronic Health Records (EHRs) contain rich longitudinal data valuable for predictive modeling.
Purpose of the Study:
- To evaluate the performance of deep learning transformer models, specifically BERT and XLNet, in predicting six-month mortality risk in cardiovascular disease patients.
- To compare the efficacy of XLNet against BERT for mortality prediction using EHR data.
- To explore the application of advanced transformer architectures for analyzing sequential clinical event data in EHRs.
Main Methods:
- Utilized EHR data from over 23,000 cardiac patients.
- Formulated patient histories as time series of clinical events to capture temporal dependencies.
- Trained and compared two transformer models: BERT and XLNet, for a six-month mortality prediction task.
- Evaluated model performance using the area under the receiver operating characteristic curve (AUC) and recall.
Main Results:
- Both BERT and XLNet demonstrated predictive capabilities, achieving average AUCs of 75.5% and 76.0%, respectively.
- XLNet outperformed BERT in recall by 9.8%, indicating a superior ability to identify patients at higher risk of mortality.
- This study represents the first known application of XLNet to EHR data for mortality prediction.
Conclusions:
- Transformer models, particularly XLNet, show significant promise for predicting mortality risk in cardiovascular disease patients using EHR data.
- XLNet's enhanced recall suggests its potential for improving early identification of high-risk individuals.
- Further research leveraging advanced deep learning on EHRs can enhance chronic disease management and patient care.
Abstract:
With over 17 million annual deaths, cardiovascular diseases (CVDs) dominate the cause of death statistics. CVDs can deteriorate the quality of life drastically and even cause sudden death, all the while inducing massive healthcare costs. This work studied state-of-the-art deep learning techniques to predict increased risk of death in CVD patients, building on the electronic health records (EHR) of over 23,000 cardiac patients. Taking into account the usefulness of the prediction for chronic disease patients, a prediction period of six months was selected. Two major transformer models that rely on learning bidirectional dependencies in sequential data, BERT and XLNet, were trained and compared. To our knowledge, the presented work is the first to apply XLNet on EHR data to predict mortality. The patient histories were formulated as time series consisting of varying types of clinical events, thus enabling the model to learn increasingly complex temporal dependencies. BERT and XLNet achieved an average area under the receiver operating characteristic curve (AUC) of 75.5% and 76.0%, respectively. XLNet surpassed BERT in recall by 9.8%, suggesting that it captures more positive cases than BERT, which is the main focus of recent research on EHRs and transformers.
