Prediction of Continuous Emotional Measures through Physiological and Visual Data

Area of Science:

• Affective computing
• Human-computer interaction
• Machine learning for mental health

Background:

• Affective state prediction is crucial for personalized digital experiences.
• Previous work focused on physiological data like electrodermal activity (EDA) and electrocardiogram (ECG).
• Virtual reality (VR) offers potential for cognitive remediation but requires adaptive environments to prevent user discouragement.

Purpose of the Study:

• To predict arousal and valence values from physiological and video data.
• To develop adaptive VR environments for cognitive remediation in mental health disorders.
• To improve prediction models by enhancing preprocessing and incorporating novel feature selection and decision fusion.

Main Methods:

• Utilized physiological (EDA, ECG) and video recordings as data sources.
• Implemented advanced machine learning models combined with refined preprocessing steps.
• Incorporated novel feature selection and decision fusion techniques.
• Evaluated the approach on the RECOLA dataset.

Main Results:

• Achieved high prediction accuracy with a concordance correlation coefficient (CCC) of 0.996 for arousal and 0.998 for valence using physiological data.
• Outperformed existing state-of-the-art methods on the RECOLA dataset.
• Demonstrated the effectiveness of combining multiple data sources and advanced machine learning.

Conclusions:

• Advanced machine learning techniques can accurately predict affective states from diverse data.
• The developed models show significant potential for personalizing VR environments for mental health applications.
• This approach can enhance cognitive remediation by adapting VR experiences to user emotional states, reducing discouragement.