Related Experiment Video
Updated: Jan 18, 2026
Real-Time Cardiac Mapping with a Noninvasive Imageless Electrocardiographic Imaging System
Published on: April 11, 2025
Bimodal ECG and PCG Cardiovascular Disease Detection: Exploring the Potential and Modality Contribution
Alessia Calzoni1,2, Mattia Savardi3, Marco Silvestri4
1University of Brescia, Department of Information Engineering, Via Branze 38, Brescia, 25123, Italy. alessia.calzoni@unibs.it.
Insights
This study introduces a novel deep learning model combining electrocardiogram (ECG) and phonocardiogram (PCG) signals for earlier cardiovascular disease (CVD) detection. The bimodal approach significantly improves diagnostic accuracy compared to single-modality methods.
Area of Science:
- Biomedical Engineering
- Artificial Intelligence in Medicine
- Cardiology
Background:
- Early detection of cardiovascular diseases (CVDs) is vital for patient outcomes and reducing healthcare costs.
- Electrocardiograms (ECGs) and phonocardiograms (PCGs) are cost-effective, non-invasive tools for CVD screening.
- Limited availability of bimodal (ECG+PCG) datasets hinders the development of advanced diagnostic models.
Purpose of the Study:
- To develop and evaluate a novel bimodal deep learning model integrating ECG and PCG signals for enhanced early CVD detection.
- To address the challenge of limited bimodal data by leveraging pre-trained models and publicly available unimodal datasets.
- To interpret the model's decision-making process and visualize feature separation between normal and pathological samples.
Main Methods:
- A bimodal deep learning architecture was proposed, featuring a late fusion of a fine-tuned audio-pre-trained CNN (for PCG) and a 1D-CNN (for ECG).
- The PCG branch was fine-tuned using all available unimodal PCG datasets.
- The model was evaluated on an augmented version of the MITHSDB dataset, employing explainability techniques and UMAP for visualization.
Main Results:
- The bimodal model achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) of 96.4%, outperforming ECG-only (approx. 93.4%) and PCG-only (approx. 85.4%) models.
- Explainability methods quantified the contributions of electrical and acoustic features.
- UMAP visualization demonstrated clear separation between normal and pathological cardiac samples.
Conclusions:
- Combining ECG and PCG signals via a bimodal deep learning approach significantly enhances early CVD detection accuracy.
- Explainability and visualization techniques provide valuable insights into the model's performance and feature contributions.
- The study highlights the potential of multimodal data fusion for CVD diagnosis and emphasizes the need for larger, diverse bimodal datasets.
Abstract:
Early detection of cardiovascular diseases (CVDs) is crucial for improving patient outcomes and alleviating healthcare burdens. Electrocardiograms (ECGs) and phonocardiograms (PCGs) offer low-cost, non-invasive, and easily integrable solutions for preventive care settings. In this work, we propose a novel bimodal deep learning model that combines ECG and PCG signals to enhance the early detection of CVDs. To address the challenge of limited bimodal data, we fine-tuned a Convolutional Neural Network (CNN) pre-trained on large-scale audio recordings, leveraging all publicly available unimodal PCG datasets. This PCG branch was then integrated with a 1D-CNN ECG branch via late fusion. Evaluated on an augmented version of MITHSDB, currently the only publicly available bimodal dataset, our approach achieved an AUROC of 96.4%, significantly outperforming ECG-only and PCG-only models by approximately 3%pts and 11%pts, respectively. To interpret the model's decisions, we applied three explainability techniques, quantifying the relative contributions of the electrical and acoustic features. Furthermore, by projecting the learned embeddings into two dimensions using UMAP, we revealed clear separation between normal and pathological samples. Our results conclusively demonstrate that combining ECG and PCG modalities yields substantial performance gains, with explainability and visualization providing critical insights into model behavior. These findings underscore the importance of multimodal approaches for CVDs diagnosis and prevention, and strongly motivate the collection of larger, more diverse bimodal datasets for future research.
More Related Videos
Related Concept Videos
01:25Correlation between ECG and Cardiac Cycle
A cardiac action potential originates in the SA node and spreads throughout the atria and the AV node in approximately 0.03 seconds. This results in the P wave in an ECG and triggers atrial contraction. The action potential is then briefly slowed at the AV node, allowing the atria to contract and fill the ventricles with blood before...
01:29Electrocardiogram
Three major waveforms are present in a typical ECG recording: the P wave, the QRS complex, and...
01:17Imaging Studies for Cardiovascular System I:Echocardiography
Indications: Echocardiography is utilized to diagnose heart failure, valve disorders, and myocardial infarction. It also assesses cardiac structures' size, shape, and motion,...
01:30Pulse rhythm
Conversely, an irregular pulse pattern is termed dysrhythmia, stemming from disruptions in cardiac...
01:20Imaging Studies for Cardiovascular System II:Types of Echocardiography
Types of Echocardiography
Transthoracic Echocardiography (TTE)
TTE is the most common type of echocardiogram which involves placing a transducer on the patient's chest, emitting sound waves to create heart images. TTE is invaluable for evaluating the heart's size, structure, and motion, making it particularly useful for...
01:24Bode Plots Construction