Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

Area of Science:

• Digital Health
• Computational Psychiatry
• Machine Learning in Healthcare

Background:

• Mental health (MH) disorders present diagnostic challenges due to complex symptoms and comorbidities, risking underdiagnosis.
• Machine learning (ML) offers potential solutions, but effective strategies for passively collected, multimodal data are underexplored.

Approach:

• Systematic review of 184 studies on ML for MH disorder detection using passively sensed multimodal data (audio, video, social media, smartphones, wearables).
• Assessed feature extraction, feature fusion, and ML methodologies, focusing on non-intrusive data collection and natural behavior capture.

Key Points:

• Modality-specific features show varying correlations influenced by individual contexts, demographics, and personalities.
• Neural networks are increasingly used for fusion and as ML algorithms, effectively handling high-dimensional data and cross-modality relationships.
• Passive sensing offers a promising avenue for naturalistic behavioral data collection.

Conclusions:

• This review provides a taxonomy of methodological approaches for multimodal MH disorder detection, guiding future research.
• Informed selection of data sources and ML methods is crucial for advancing the accuracy and scope of digital mental health diagnostics.