Multimodal machine learning for video based single question mental health assessment.

Area of Science:

• Computational psychiatry
• Digital mental health
• Multimodal AI

Background:

• Increasing demand for mental health screening.
• Need for efficient, multi-condition assessment tools.
• Current screening methods can be time-consuming and burdensome.

Purpose of the Study:

• To evaluate the efficacy of a single video question for predicting depression, anxiety, and trauma.
• To assess the feasibility of using text and voice analysis for mental health screening.
• To reduce patient burden and assessment time in clinical settings.

Main Methods:

• Developed a multimodal model integrating MPNet for text and HuBERT for voice prosody.
• Trained the model on data from 2420 participants.
• Utilized a single video question to capture responses for analysis.

Main Results:

• The model accurately predicted self-reported depression (PHQ-9), anxiety (GAD-7), and trauma (PCL-5).
• Achieved a 64.6% reduction in assessment time (78.4s vs 221.7s).
• Demonstrated strong performance and demographic consistency across age, gender, and race/ethnicity.

Conclusions:

• A single video question is a feasible and efficient method for multi-condition mental health screening.
• This approach significantly reduces assessment time while maintaining accuracy.
• Video-based screening shows high participant acceptance (98.6% willingness).