A multimodal fusion network for heart sound abnormality detection and classification

Area of Science:

• Cardiology
• Artificial Intelligence
• Signal Processing

Background:

• Accurate cardiac function assessment from heart sounds is hindered by noise and variable heart rates.
• Existing methods often require complex cardiac-cycle segmentation, limiting practical application.

Purpose of the Study:

• To develop a fully end-to-end (E2E) deep learning framework for direct diagnostic information extraction from raw heart sound recordings.
• To enable accurate cardiac abnormality detection and classification.

Main Methods:

• Proposed HS-MMNet, an E2E multi-modal deep learning framework utilizing Convolution and Transformer Heads.
• Employed multi-atrous spatial pyramid, channel-spatial attention, and a novel Multi-Hypothesis Cross-Attention (MH-CA) module for noise suppression and feature fusion.
• Processed 1-D waveforms and Log-Mel spectrograms from 2.5-second segments.

Main Results:

• Achieved state-of-the-art (SOTA) performance on the PhysioNet/CinC Challenge 2016 dataset with 94.80% accuracy.
• Attained 99.60% macro-averaged precision, recall, and F1-score on a five-class Yaseen dataset, demonstrating high diagnostic accuracy.
• Outperformed all previously reported methods on benchmark datasets.

Conclusions:

• HS-MMNet advances automated physiological measurement from heart sounds, eliminating the need for cardiac cycle detection.
• Offers a practical and scalable solution for cardiovascular screening in primary care and low-resource settings.
• Achieved SOTA diagnostic performance, highlighting its potential for widespread clinical adoption.