Remote Estimation of Video-Based Vital Signs in Emotion Invocation Studies

Area of Science:

• Physiology
• Psychology
• Computer Vision

Background:

• Emotion recognition is crucial for human-computer interaction.
• Vital signs offer objective physiological markers of emotional states.
• Non-invasive, cost-effective methods for vital sign monitoring are needed.

Purpose of the Study:

• To investigate the impact of imitated and video-induced emotions on respiratory and pulse rates.
• To assess the feasibility of extracting vital sign data from low-cost camera footage.
• To compare the efficacy of respiratory rate versus pulse rate in differentiating emotions.

Main Methods:

• Participants experienced various emotions through imitation and video stimuli.
• Respiratory and pulse rates were monitored during emotional induction.
• Analysis of vital sign data extracted from video recordings using cost-effective cameras.
• Evaluation of facial expression recognition accuracy from video.

Main Results:

• Respiratory rate showed greater potential than pulse rate for distinguishing certain emotions.
• A positive emotion induced a significant increase (>1.8 bpm) in respiratory rate compared to neutral states (89% of observations).
• Facial expression analysis from video was often insufficient for accurate emotion classification (11.4% accuracy).

Conclusions:

• Respiratory rate is a promising physiological indicator for emotion detection.
• Cost-effective camera-based vital sign monitoring is feasible but requires further refinement.
• Integrating physiological data with other modalities may improve emotion recognition accuracy.