A 1D Snoring Waveform and 2D Composite Acoustic Feature Graph-Based Multi-Modal Fusion Network for Obstructive Sites...

Area of Science:

• Biomedical Engineering
• Otorhinolaryngology
• Signal Processing

Background:

• Accurate localization of obstructive sites in the upper airway is crucial for diagnosing and treating sleep-related breathing disorders.
• Snoring, a dynamic acoustic signal, offers a non-invasive method for identifying obstruction sites but existing methods often use limited information or single modalities.
• Current approaches may overlook complementary data by analyzing either 1D signals or 2D images separately.

Purpose of the Study:

• To develop and validate a multi-modal framework for accurate obstructive site recognition in the upper airway using snoring sounds.
• To integrate information from 1D snoring waveforms and 2D Composite Acoustic Feature Graphs (CAF-Graph) for a more comprehensive analysis.
• To improve the diagnostic capabilities for sleep-related breathing disorders through enhanced snoring analysis.

Main Methods:

• Proposed a multi-modal framework combining 1D snoring waveform analysis and 2D CAF-Graph representation.
• Utilized neural networks to learn discriminative representations from the 1D snoring waveform, capturing fine time structures.
• Developed a 2D CAF-Graph emphasizing spatio-temporal and physiological-acoustic characteristics by concatenating Prosodic, Formant, Spectral, and Cepstral features.
• Employed a multi-modal fusion network (BMFNet) to integrate information from both modalities.

Main Results:

• The multi-modal framework achieved 81.2% Accuracy, 86.8% Weighted Average Precision, 81.2% Weighted Average Recall, and 82.3% Weighted Average F1-Score on a clinical dataset.
• The classification task included three categories: upper airway obstruction, lower airway obstruction, and absence of snoring.
• Demonstrated the effectiveness of integrating multi-modal features for improved snoring analysis and obstructive site recognition.

Conclusions:

• The proposed multi-modal framework effectively utilizes both 1D snoring waveforms and 2D CAF-Graphs for accurate obstructive site recognition.
• This approach provides a more comprehensive perspective by integrating independent and interactive information between single-modal features.
• The findings offer a novel insight and a potentially valuable tool for the non-invasive diagnosis of sleep-related breathing disorders.