CEMTNet: a cognitive emotion modulated network for multimodal depression detection

Early depression detection is a critical task in public health, making automatic depression identification increasingly important. Existing multimodal methods face challenges of cross-modal emotional inconsistency, subtle emotional cue extraction, and dynamic emotional modeling. To address these issues, we propose Cognitive-Emotion-Modulated Trend Network (CEMTNet) for robust multimodal depression recognition from audio-text data. CEMTNet integrates three modules: Cognitive Consistency Inference Mechanism (CCIM) dynamically adjusts fusion weights based on estimated emotional consistency; Emotion-Modulated Structured Attention (EMSA) enhances fine-grained emotional cue extraction using emotion-aware multi-head attention; and Contrastive Emotion Trend Modeling (CE-TM) integrates contrastive learning with temporal trend modeling.The experimental results on the DAIC-WOZ and EATD-Corpus datasets demonstrate that CEMTNet consistently outperforms existing state-of-the-art methods, achieving an accuracy of 0.92, an F1-score of 0.92, and a recall of 0.93. The ablation studies and case analyses further confirm the effectiveness and robustness of the proposed modules. The source code can be found at https://github.com/ghy-yhg/CEMTNet.