Hypergraph-Based Audio-Visual Fusion for Obstructive Sleep Apnea Severity Estimation During Wakefulness

Area of Science:

• Biomedical Engineering
• Sleep Medicine
• Artificial Intelligence in Healthcare

Background:

• Obstructive sleep apnea (OSA) is linked to significant psychophysiological impairments.
• Current methods for OSA severity estimation using unimodal data (speech, images) have limitations.
• Integrating diverse physiological signals may enhance diagnostic capabilities.

Purpose of the Study:

• To develop a novel multimodal fusion framework for improved OSA severity estimation during wakefulness.
• To leverage audio-visual data and heart rate signals for more accurate OSA diagnosis.
• To overcome the limitations of unimodal data in current OSA assessment tools.

Main Methods:

• Proposed a hypergraph-based multimodal fusion framework (HMFusion).
• Employed Long Short-Term Memory (LSTM) encoders for audio-visual and rPPG-derived heart rate data.
• Utilized a hypergraph neural network to model cross-modal interactions for OSA severity prediction.

Main Results:

• Achieved high performance in OSA severity estimation across different Apnea-Hypopnea Index (AHI) thresholds.
• Reported Area Under the ROC Curve (AUC) scores ranging from 85.29% to 88.26%.
• Obtained F1-scores between 85.30% and 92.91%, outperforming existing state-of-the-art methods.

Conclusions:

• Psychophysiological data, when fused multimodally, significantly enhances OSA severity estimation during wakefulness.
• The HMFusion framework demonstrates a promising new direction for clinical research and non-invasive OSA diagnosis.
• This approach offers potential for more accurate and accessible OSA assessment in clinical settings.