Self-Supervision-Enabled Compounded Multi-Modal Feature-Learning Network for Classifying Depressive States with...

Area of Science:

• Computational psychiatry and affective computing.
• Development of advanced machine learning models for mental health assessment.

Background:

• Depression is a widespread mental health issue impacting daily life.
• Wearable monitoring offers continuous assessment but faces privacy and accuracy challenges.
• Existing methods struggle to capture complex emotional nuances.

Purpose of the Study:

• To propose a Self-Supervision-Enabled Compounded Multi-Modal Feature-Learning Network (S2-CFL) for accurate depressive state classification.
• To integrate wearable sensor data with psychological self-reports for comprehensive analysis.
• To enhance the understanding of emotional states and depressive patterns.

Main Methods:

• Utilized a Twin-Path Encoder-Decoder Network (TP-EDN) for temporal feature extraction from raw signals.
• Employed a Densely Connected Convolution Pyramidal Transformer Network (DC2-PTN) for spatial representation learning.
• Integrated a fusion mechanism and a Fine-Grained Emotion Classification Network (FGECN) for multi-modal analysis and classification.

Main Results:

• The S2-CFL framework demonstrated improved classification performance for depressive states.
• The multi-modal approach provided interpretable insights into emotional and depressive patterns.
• The system successfully predicted depressive states, valence, and arousal levels.

Conclusions:

• The proposed S2-CFL offers a robust solution for mental health monitoring using wearable technology.
• Multi-modal feature learning significantly enhances the accuracy and depth of depression assessment.
• This approach holds promise for personalized mental healthcare and early intervention strategies.