MHCL: Multi-modal Hierarchical Contrastive Learning for Physiological Signal-Based Vigilance Detection

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Vigilance detection is crucial for safety-critical applications like driver monitoring.
• Individual variability and weak physiological signal characteristics pose challenges for accurate vigilance state classification.
• Existing methods struggle with inter-subject variability and robust multi-modal fusion.

Purpose of the Study:

• To develop a novel multi-modal framework (MHCL) integrating EEG and EOG signals for enhanced vigilance detection.
• To address challenges in individual variability and improve generalization across subjects.
• To achieve superior performance compared to unimodal and existing multi-modal approaches.

Main Methods:

• Proposed MHCL framework utilizing hierarchical contrastive learning for representation alignment.
• Dual-stage fusion incorporating cross-modal attention and ensembled classification.
• Adversarial domain adaptation with label smoothing to mitigate inter-subject variability.

Main Results:

• Achieved 96.9% accuracy in subject-dependent settings and 82.0% in cross-subject evaluations on the SEED-VIG dataset.
• Significantly outperformed existing unimodal and multi-modal baseline methods.
• Visualization confirmed the model's ability to learn semantically aligned and subject-invariant representations.

Conclusions:

• MHCL offers a robust solution for vigilance detection by effectively fusing EEG and EOG signals.
• The framework successfully addresses key challenges in multi-modal fusion and cross-subject generalization.
• The developed model provides a significant advancement for safety-sensitive applications requiring reliable vigilance monitoring.