DynMultiDep: A Dynamic Multimodal Fusion and Multi-Scale Time Series Modeling Approach for Depression Detection

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Psychiatry

Background:

• Depression is a major global public health concern.
• Current multimodal depression detection methods struggle with integrating long-term and short-term data patterns and adapting to varying data complexities.
• Existing static fusion strategies limit the dynamic adaptation to complementary and redundant information across different data modalities.

Purpose of the Study:

• To propose a dynamic multimodal depression detection framework, DynMultiDep.
• To address the limitations of insufficient integration of global patterns and local fluctuations in long-sequence modeling.
• To overcome the challenge of static fusion strategies in multimodal depression detection.

Main Methods:

• Developed DynMultiDep, a framework combining multi-scale temporal modeling and adaptive fusion.
• Introduced the Multi-scale Temporal Experts Module (MTEM) using Mamba and Transformers for long-term and short-term feature extraction.
• Implemented the Dynamic Multimodal Fusion module (DynMM) for adaptive modality selection and cross-modal interaction.

Main Results:

• DynMultiDep demonstrated superior performance compared to existing state-of-the-art methods.
• The framework achieved enhanced detection accuracy on two large-scale depression datasets.
• The dynamic fusion mechanism effectively adapted to input characteristics, optimizing information integration.

Conclusions:

• DynMultiDep offers a significant advancement in dynamic multimodal depression detection.
• The proposed MTEM and DynMM modules effectively address key bottlenecks in current methods.
• This framework holds promise for improving the accuracy and adaptability of AI-driven mental health diagnostics.