End-to-end multimodal clinical depression recognition using deep neural networks: A comparative analysis.

Area of Science:

• Computational psychiatry
• Machine learning for healthcare
• Multimodal data analysis

Background:

• Major Depressive Disorder (MDD) is a prevalent and disabling condition.
• Deep learning models for multimodal depression recognition are emerging.
• A comparative analysis of these models is lacking in current literature.

Purpose of the Study:

• To provide an up-to-date literature overview of multimodal depression recognition.
• To perform an extensive comparative analysis of deep learning architectures for depression recognition.
• To evaluate fusion strategies for combining audio, visual, and textual features.

Main Methods:

• Literature review of multimodal depression recognition.
• Comparative analysis of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) for audio features.
• Investigation of early-level and model-level fusion of audio, visual, and textual features using CNN and LSTM architectures.

Main Results:

• LSTM-based audio features slightly outperformed CNNs for binary depression classification (66.25% vs. 65.60%).
• Model-level fusion of audio and visual features using LSTM achieved the highest accuracy (77.16%) for binary classification.
• The best-performing LSTM model achieved 95.38% accuracy for binary detection and a normalized RMSE of 0.1476 for severity prediction using Leave-One-Subject-Out cross-validation.

Conclusions:

• LSTM-based architectures are superior to CNNs for capturing temporal dynamics in multimodal depression recognition.
• Model-level fusion of audio and visual features with LSTM yields the best performance.
• The developed model is efficient for real-world clinical applications, detecting depression in under 8 seconds with low computational cost.