Applying masked autoencoder-based self-supervised learning for high-capability vision transformers of...

Area of Science:

• Artificial Intelligence in Medicine
• Cardiology
• Machine Learning

Background:

• Generalizing deep neural network (DNN) algorithms to diverse populations is a key challenge in medical AI.
• Limited data availability often hinders the performance of DNN models for medical tasks like electrocardiography (ECG) analysis.

Purpose of the Study:

• To enhance the performance of 12-lead ECG analysis models using self-supervised learning with masked autoencoders (MAEs).
• To improve the detection of left ventricular systolic dysfunction (LVSD) using limited ECG data.

Main Methods:

• Pretrained Vision Transformer (ViT) models using MAE by reconstructing masked ECG data.
• Fine-tuned the MAE-based pre-trained model on ECG-echocardiography data for LVSD detection.
• Evaluated model performance using multi-center external validation data (n=229,439) and the AUROC metric.

Main Results:

• MAE-based ECG models demonstrated significantly higher performance than other DNN models across external validation cohorts (AUROC: 0.913-0.962 for LVSD, p < 0.001).
• Performance improvements were observed with increased model capacity and training data.
• The MAE-based model maintained high performance on the PTB-XL ECG benchmark dataset.

Conclusions:

• Self-supervised learning with MAEs effectively develops high-performance ECG analysis models, even with limited data.
• The proposed MAE-based approach enhances the generalization and accuracy of AI in cardiovascular diagnostics.