Related Experiment Videos
MMMuLNet: Multimodal Mutual Deep learning Framework for the Pediatric Congenital Heart Disease Detection
Insights
A new multimodal framework accurately detects congenital heart disease (CHD) in children using heart sounds (PCG), electrical activity (ECG), and symptoms. This approach offers a scalable solution for early CHD detection, especially in low-resource settings.
Area of Science:
- Biomedical Engineering
- Cardiology
- Artificial Intelligence in Medicine
Background:
- Congenital heart disease (CHD) is a leading cause of infant mortality globally.
- Echocardiography, the gold standard for CHD diagnosis, is limited in low- and middle-income countries (LMICs) due to cost and expertise requirements.
- There is a critical need for accessible, reliable diagnostic tools for pediatric CHD in resource-limited environments.
Purpose of the Study:
- To develop and validate a multimodal framework for accurate pediatric CHD detection.
- To integrate phonocardiograms (PCG), electrocardiograms (ECG), and clinical symptoms for enhanced diagnostic performance.
- To ensure the framework's robustness under real-world conditions, including degraded signal quality.
Main Methods:
- A multimodal framework combining PCG, ECG, and clinical symptoms was developed.
- Features were extracted from PCG and ECG using a pretrained audio foundation model; symptoms were selected using SHAP.
- A fusion network with modality dropout and mutual learning was employed for robust integration and handling of missing data.
Main Results:
- The full multimodal system achieved 94.5% accuracy, 94.2% sensitivity, 95.6% specificity, and 96% AUROC.
- Individual modalities showed PCG alone (90% accuracy), ECG alone (79%), and symptoms alone (73%).
- The multimodal framework maintained 88% accuracy and 87% AUROC even with low-quality PCG and ECG signals, outperforming unimodal approaches.
Conclusions:
- Integrating PCG, ECG, and clinical symptoms provides an accurate and resilient method for pediatric CHD screening.
- This multimodal approach offers a practical and scalable solution for early CHD detection in LMICs.
- The framework demonstrates significant potential for improving diagnostic accessibility and patient outcomes in underserved regions.
Abstract:
Congenital heart disease (CHD) is the most common birth defect and a major cause of infant morbidity and mortality worldwide. While echocardiography remains the diagnostic gold standard, its cost and reliance on expert interpretation limit availability in low- and middle-income countries (LMICs). This underscores the need for scalable, affordable, and robust approaches that can operate reliably under real-world signal degradation and low-quality recording conditions. In this study, we present a multimodal framework for pediatric CHD detection that integrates phonocardiograms (PCG), electrocardiograms (ECG), and clinical symptoms. Features from PCG and ECG were extracted using a pretrained foundation audio model, while key symptoms were selected using SHAP (SHapley Additive exPlanations) and encoded as structured embeddings. These modalities were combined in a fusion network with modality dropout to handle missing inputs, and mutual learning was applied to promote knowledge sharing across unimodal and multimodal branches. The framework was validated on a dataset of 751 pediatric patients in Bangladesh, comprising 3,435 synchronized PCG-ECG recordings with expert-confirmed labels. In 10-fold patient-wise cross validation, PCG alone achieved 90% accuracy, ECG alone 79%, and symptoms alone 73%. Combining PCG and ECG improved accuracy to 94%, while the full multimodal system reached 94.5% accuracy, 94.2% sensitivity, 95.6% specificity, and an AUROC of 96%. Importantly, even under degraded signal conditions when both PCG and ECG were of low quality, the multimodal framework maintained an accuracy of 88% and an AUROC of 87%, demonstrating superior performance compared to the unimodal models. These findings demonstrate that integrating PCG, ECG, and symptoms enables accurate, resilient CHD screening, offering a practical pathway for scalable early detection in LMICs.