Measuring depression severity based on facial expression and body movement using deep convolutional neural network.

Area of Science:

• Artificial Intelligence
• Clinical Psychology
• Biomedical Engineering

Background:

• Current depression severity evaluations rely on subjective, time-consuming methods like scales and interviews.
• Existing AI approaches often use single-modal data, limiting prediction accuracy.
• There's a need for objective, real-time tools to assess major depressive disorder (MDD) severity.

Purpose of the Study:

• To develop and validate a multi-modal AI model for real-time depression severity measurement.
• To assess the model's accuracy using expression and action features from video data.
• To evaluate the model's utility in tracking depression changes during treatment.

Main Methods:

• A multi-modal deep convolutional neural network (CNN) was developed.
• Facial expression and body movement data were extracted from videos.
• Behavioral Depression Degree (BDD) metrics, combining expression and action entropy, were established.

Main Results:

• The multi-modal approach significantly improved evaluation accuracy compared to single-modal methods.
• BDD showed over 74% Pearson similarity with established depression and anxiety scales (SDS, SAS, HAMD).
• BDD effectively tracked depression severity changes consistent with clinical evaluations during treatment.

Conclusions:

• The developed BDD metric accurately reflects current depression state and treatment trends.
• This AI-driven, multi-modal approach offers a potential automatic auxiliary tool for MDD diagnosis and management.
• Real-time, objective assessment of depression severity is feasible using AI and patient behavioral data.