SBT-Net: a tri-cue guided multimodal fusion framework for depression recognition

Area of Science:

• Computational psychiatry
• Artificial intelligence in healthcare
• Multimodal machine learning

Background:

• Early depression detection is crucial for public health.
• Current multimodal methods face challenges like incomplete data, semantic inconsistencies, and fluctuating emotional states.
• Robust depression detection requires advanced analytical frameworks.

Purpose of the Study:

• To propose SBT-Net, a novel Semantic-Bias-Trend guided framework for robust depression detection using audio and text data.
• To address limitations of existing multimodal depression detection methods.
• To improve the accuracy and reliability of automated depression assessment.

Main Methods:

• Developed SBT-Net, incorporating a semantically guided cross-modal gating (SGCMG) mechanism for feature filtering.
• Integrated a bias-guided tensor product attention (BG-TPA) mechanism for enhanced inter-modal fusion and alignment.
• Utilized an emotion trend modeling (ETM) module to capture temporal dynamics of depressive states.

Main Results:

• SBT-Net achieved 93.0% accuracy, 0.93 F1 score, and 0.92 recall on benchmark datasets (DAIC-WOZ, EATD-Corpus).
• Performance surpassed competitive baseline models across multiple evaluation metrics.
• Ablation studies confirmed the significant contributions of individual and combined modules.

Conclusions:

• The proposed SBT-Net framework demonstrates superior performance in multimodal depression detection.
• Integrating semantic guidance, bias-aware fusion, and emotional trend modeling enhances robustness.
• Findings suggest a promising direction for advancing automated mental health monitoring solutions.