[A cross-subject stress recognition method integrating bidirectional cross-modal multi-path fusion and adaptive adversarial learning]
Related Experiment Videos
Ping Wang1,2, Minyu Zhu1, Yongxin Chou3
1School of Microelectronics Industry-education Integration, Lanzhou University of Technology, Lanzhou 730050, P. R. China.
Abstract:
Multimodal physiological signal-based stress recognition faces challenges in cross-modal correlation modeling and limited cross-subject generalization. To address these issues, this paper proposes a cross-modal interaction and adversarial learning-based stress recognition method. First, a bidirectional cross-modal-cross-view multi-path fusion module is designed, which enables multimodal feature interaction and aggregation through six cross-modal pairwise combinations, multi-path modeling, and adaptive gating. Furthermore, a subject adversarial learning mechanism based on state-adaptive gradient reversal is proposed to dynamically adjust the adversarial strength and suppress subject-specific information in the fused representations. Meanwhile, a joint optimization strategy integrating classification, six-view contrastive learning, and subject adversarial learning is constructed to collaboratively enhance stress discriminability, cross-view consistency, and subject invariance. Experiments conducted on the publicly available Wearable Stress and Affect Detection (WESAD) dataset demonstrate that the proposed method achieves a cross-subject recognition accuracy of 91.58%. Comparative and ablation studies further validate its effectiveness in cross-subject stress recognition. This study provides a new solution for cross-modal information fusion and subject-invariant representation learning in multimodal physiological signal-based stress recognition and offers methodological references for the development of wearable intelligent stress monitoring systems.