End-to-end multimodal system for depression detection from online recordings

Area of Science:

• Artificial Intelligence
• Machine Learning
• Mental Health Technology

Background:

• Depression is a widespread "civilizational disease" requiring innovative detection methods.
• Current screening methods can be time-consuming and may lack objective, real-time data.
• Social media offers a rich, albeit complex, source of behavioral data for analysis.

Purpose of the Study:

• To develop and validate a novel approach for depression detection using multimodal social media content analysis.
• To leverage advanced machine learning and deep learning models for accurate depression classification.
• To enable early detection and intervention for mental health support.

Main Methods:

• Utilized audio-visual analysis of social media content, extracting over 605 time-domain features.
• Employed machine learning and deep learning models for classifying depression.
• Integrated face analysis, emotion recognition neural networks, and speech processing.

Main Results:

• Achieved an F1-score of 0.77, outperforming existing state-of-the-art models.
• Demonstrated the efficacy of multimodal (audio-visual) analysis for non-verbal behavioral assessment.
• Developed a system capable of real-time analysis for efficient depression screening.

Conclusions:

• The proposed approach offers a significant advancement in mental health technology for depression detection.
• Real-time, objective measurements can complement traditional diagnostic methods and support mental health professionals.
• This technology has the potential for broad societal impact through early intervention and accessible mental health resources.