Decoding depression with computer vision-assisted analysis of synchronized facial expressions

Facial expressions convey nonverbal signals of internal emotional states and hold potential as important markers for affective disorders such as depression.

Here, we adopted a naturalistic paradigm and combined with an inter-subject correlation (ISC) framework to identify diagnostic cues for depression from facial expression synchronization patterns while naturally responding to emotionally charged videos. Using timeseries activity patterns of action units (AUs) extracted from facial expressions of emotion, we developed models for detecting depression with AU-ISC vector as key features.

Resulting models successfully detected depression at 72-90 % accuracy. In contrast, models using mean AU activity exhibited poor performance, showcasing the diagnostic utility of the AU-ISC models.

These findings underscore the potential utility of facial expression data in advancing objective and clinically meaningful diagnostic tools for depression. More broadly, the present study highlights a new direction for emotion research, utilizing facial expression dynamics as a complementary tool alongside established self-report measures.