WavFace: A Multimodal Transformer-Based Model for Depression Screening.

Area of Science:

• Artificial Intelligence
• Mental Health Technology
• Computational Psychiatry

Background:

• Depression is a widespread mental health disorder with significant health and economic impacts.
• Current depression detection methods can be costly and challenging.
• Deep learning (DL) models show potential for depression screening using clinical interview videos.

Purpose of the Study:

• To develop a multimodal deep learning model for accurate depression screening.
• To address challenges in modality representation, alignment, fusion, and small sample sizes in DL models.
• To propose WavFace, a novel model integrating audio and temporal facial features.

Main Methods:

• Developed WavFace, a multimodal deep learning model using audio and temporal facial features.
• Incorporated an encoder-transformer layer for enhanced unimodal representation.
• Implemented explicit alignment and sequential/spatial self-attention for modality fusion.
• Modeled after clinical observations of visual and vocal cues in mental health assessments.

Main Results:

• WavFace demonstrated superior performance compared to existing unimodal and multimodal DL models.
• Achieved a balanced accuracy of 0.81 in depression screening using a single interview question.
• Successfully integrated audio and visual data for robust mental health screening.

Conclusions:

• WavFace offers a valuable and effective modeling approach for audio-visual mental health screening.
• The model's ability to process multimodal data enhances depression detection accuracy.
• This research contributes to advancing AI-driven tools for mental health assessment.