Detecting stress from videos via intra-subject and inter-subject learning

Area of Science:

• Psychology
• Computer Science
• Biomedical Engineering

Background:

• Mental stress is a significant public health concern.
• Accurate video-based stress detection is difficult due to individual variability in physiological and expressive responses.
• Existing methods often fail to account for personalized stress baselines.

Purpose of the Study:

• To develop a novel two-level learning framework for robust video-based stress detection.
• To model stress as a personalized deviation from an individual's physiological baseline, grounded in allostasis theory.
• To improve stress detection accuracy by incorporating intra-subject and inter-subject adaptive mechanisms.

Main Methods:

• Proposed a two-level learning framework incorporating allostasis theory.
• Introduced CalmScore, a metric based on resting heart rate variability (HRV), to establish a personalized baseline.
• Developed a Physiological Discrepancy based Representation Adaptive Modulation module for intra-subject analysis.
• Implemented an inter-subject analogical reasoning mechanism using In Context Instruction Tuning for peer-based calibration.

Main Results:

• Achieved state-of-the-art F1 scores: 96.85% on the UVSD dataset and 88.67% on the RSL dataset.
• Outperformed the strongest existing baseline methods.
• Ablation studies confirmed the necessity and contribution of each proposed component.
• Demonstrated the effectiveness of modeling individualized deviations and physiological analogy for stress detection.

Conclusions:

• The proposed framework significantly enhances the accuracy and robustness of video-based stress detection.
• Modeling stress as personalized deviations from a baseline, calibrated by HRV and peer analogy, is crucial for overcoming inter-individual variability.
• This approach offers a promising direction for developing more effective stress monitoring tools.