Deep learning-based detection of depression by fusing auditory, visual and textual clues

Area of Science:

• Artificial Intelligence
• Computational Psychiatry
• Machine Learning

Background:

• Early depression detection is vital for timely intervention.
• Automated analysis of visual, acoustic, and semantic signals is advancing with deep learning.
• Current methods for depression assessment can be time-consuming and subjective.

Purpose of the Study:

• To propose an automated depression detection model integrating visual, auditory, and textual data.
• To validate the model's performance across various scenarios, including chatbot interactions.
• To assess the generalizability of the multimodal depression detection model.

Main Methods:

• Developed a GPT-2.0 powered chatbot for symptom inquiry.
• Captured audio-video and textual data during interviews, supplemented by a brief affective interview task.
• Fused multimodal features using a multi-head cross-attention network and performed external validation.

Main Results:

• The multimodal model achieved high accuracy (AUC > 0.950, accuracy > 0.930) in internal validation.
• Exceptional performance was observed in the chatbot interview scenario (AUC = 0.999).
• External validation demonstrated good generalizability (AUC = 0.978), though with slightly reduced performance.

Conclusions:

• The multimodal AI model shows significant potential for accurate depression detection.
• Further research is needed for longitudinal studies and applicability to severe depression cases.