Related Experiment Video
Updated: May 29, 2026
05:19
Closed-Loop Neurostimulation for Biomarker-Driven, Personalized Treatment of Major Depressive Disorder
Published on: July 7, 2023
Detection of Microbehavior Intervals for Predicting Mental Health: Clinically Relevant and Advanced Multimodal
Sapir Gershov1, Charlotte E Hilberdink1, Yiwen Zhao1
1Department of Psychiatry, Grossman School of Medicine, New York University, New York, NY, United States.
Journal of Medical Internet Research
|May 27, 2026
Summary
Microbehavior intervals from nonverbal signals accurately detect psychological distress in health care workers (HCWs). This novel approach offers an objective, interpretable, and scalable complement to traditional assessments for burnout and PTSD.
Area of Science:
- Psychiatry and Behavioral Sciences
- Computer Science and Artificial Intelligence
- Human-Computer Interaction
Background:
- Health care workers (HCWs) experience high psychological distress, including burnout and posttraumatic stress disorder (PTSD), due to demanding work environments.
- Traditional self-report tools for assessing distress are limited by stigma and underreporting.
- Nonverbal behaviors, particularly fine-grained temporal fluctuations (microbehavior intervals), show promise for objective distress assessment.
Purpose of the Study:
- To evaluate if microbehavior intervals derived from nonverbal signals can improve the discrimination of psychological distress profiles in HCWs.
- To assess the utility of these intervals in identifying burnout and PTSD symptoms.
Main Methods:
- HCWs underwent emotion-eliciting interviews recorded via video.
- Computer vision models analyzed nonverbal signals (facial expressions, gaze, posture, gestures) to extract time-series data.
- An unsupervised anomaly detection model identified microbehavior intervals.
- A deep learning classifier predicted four psychological distress classes: moderate-severe burnout, subthreshold-provisional PTSD, burnout+PTSD, and resilient.
Main Results:
- The classifier achieved a macro F1-score of 0.75 and an AUC of 0.80 in predicting distress profiles.
- Ablation studies indicated that gaze and arousal-valence signals were crucial for performance.
- Explainability analysis revealed that irregularity and variability in microbehaviors were key predictors of distress.
Conclusions:
- Microbehavior intervals provide a scalable, interpretable, and annotation-free framework for detecting psychological distress from nonverbal signals.
- This fine-grained, multimodal temporal modeling captures subtle, involuntary fluctuations missed by whole-video analysis.
- The approach offers an objective, robust, and explainable complement to conventional psychometric assessments for HCWs.
