MHCL: Multi-modal Hierarchical Contrastive Learning for Physiological Signal-Based Vigilance Detection

Vigilance detection using physiological signals is critical for safety-sensitive applications such as driver monitoring. However, accurately capturing vigilance-related patterns across different individuals remains a challenge due to individual variability and weak signal characteristics. This paper presents MHCL, a novel multi-modal framework that integrates electroencephalogram (EEG) and electrooculogram (EOG) signals to enhance vigilance state classification. MHCL introduces three key innovations: (1) a hierarchical contrastive learning mechanism that aligns local and global representations within and across modalities; (2) a dual-stage fusion framework that implements feature-level interaction through cross-modal attention and decision-level integration via an ensembled classifier module; (3) an adversarial domain adaptation strategy with label smoothing to mitigate inter-subject variability and an ensembled classifier to leverage both modality-specific and fused features. Extensive experiments on the SEED-VIG dataset demonstrate leading performance, achieving 96.9% accuracy in subject-dependent settings and 82.0% in cross-subject evaluations, significantly outperforming existing unimodal and multi-modal baselines. Visualization analyses confirm the model's ability to learn semantically aligned and subject-invariant representations. This work provides a robust solution for vigilance detection by addressing key challenges in multi-modal fusion and generalization. The implementation of the proposed MHCL model is publicly available at: https://github.com/xuexiba233/MHCL.