CardioFM: A Multimodal Foundation Model for Joint ECG and PPG Representation Learning

Area of Science:

• Biomedical Engineering
• Artificial Intelligence in Healthcare
• Cardiovascular Signal Processing

Background:

• Electrocardiography (ECG) and photoplethysmography (PPG) are co-acquired but lack unified foundation models.
• Existing methods are modality-specific or domain-agnostic, failing to capture cross-modal physiological coupling.
• There is a need for a model that integrates ECG and PPG for comprehensive cardiac monitoring.

Purpose of the Study:

• To develop CardioFM, a self-supervised multimodal foundation model for joint ECG and PPG analysis.
• To learn unified representations from diverse patient data for cross-context transfer.
• To demonstrate the model's utility in cardiovascular disease classification, hemodynamic trending, and intensive care alarm reduction.

Main Methods:

• Developed CardioFM using bidirectional cross-modal attention and adaptive residual vector quantization.
• Pretrained the model on over 500,000 hours of ECG and PPG data from diverse settings.
• Evaluated performance on cardiovascular disease classification, QT interval estimation, pulse arrival time measurement, and false alarm reduction.

Main Results:

• Achieved an F1-score of 0.86 for cardiovascular disease classification.
• Estimated QT interval with a mean error of 20.2 ms and pulse arrival time with a mean error of 22.7 ms.
• Demonstrated superior performance in false alarm reduction and demographic inference, encoding biological characteristics without diagnostic labels.

Conclusions:

• CardioFM consolidates cardiac biosignal analysis, offering a unified framework for clinical monitoring and wearables.
• The model's cross-modal attention enhances robustness to signal degradation.
• CardioFM is compatible with edge deployment and requires no additional sensing hardware, paving the way for broader critical illness surveillance.