Transformer-based heart language model with electrocardiogram annotations.

Area of Science:

• Artificial Intelligence
• Biomedical Engineering
• Cardiology

Background:

• Atrial Fibrillation (AFIB) is a common arrhythmia characterized by irregular heart rhythms.
• Current methods for AFIB detection in electrocardiogram (ECG) processing face challenges in pattern recognition.
• Transformer-based foundation models, successful in natural language processing, offer a novel approach for complex signal analysis.

Purpose of the Study:

• To explore the efficacy of transformer-based foundation models for detecting Atrial Fibrillation (AFIB) in ECG signals.
• To adapt natural language processing transformer architectures for analyzing ECG data represented as tokenized heartbeats.
• To evaluate the performance of these models across different tokenization strategies and dataset sizes.

Main Methods:

• A transformer-based neural network architecture was employed, treating ECG segments as sequences of tokens representing heartbeat locations.
• Foundation models were initially trained on large, annotated ECG benchmark databases.
• Subsequent fine-tuning was performed on smaller datasets, with evaluation on diverse ECG datasets not used during fine-tuning.

Main Results:

• The best-performing model, utilizing 41 heartbeats as tokens, achieved an F1 score of 93.33% for AFIB detection.
• Experiments with varying token counts (41, 128, 256, 512) demonstrated the impact of tokenization granularity.
• The study confirmed that large-scale pre-trained foundation models can be effectively fine-tuned with smaller datasets for arrhythmia classification.

Conclusions:

• Transformer-based foundation models show significant promise for accurate AFIB detection in ECG processing.
• The fine-tuning approach enables efficient adaptation of powerful models to specific medical tasks with limited data.
• This research highlights the potential of foundation models as future AI-powered tools for cardiologists, improving diagnostic capabilities.