Emotion Recognition with Refined Labels for Deep Learning.

Area of Science:

• Affective computing
• Human-computer interaction
• Cognitive neuroscience

Background:

• Traditional emotion classification assigns a single label to entire trials, overlooking the dynamic nature of emotional responses.
• This fixed annotation approach can reduce classification accuracy as emotions naturally vary over time.
• Continuous emotional traces are often simplified into discrete, static labels, losing valuable temporal information.

Purpose of the Study:

• To develop a novel thresholding scheme for more accurate, temporally sensitive emotion classification.
• To improve the performance of emotion recognition systems by accounting for the duration of emotional states.
• To enhance the accuracy of long short-term memory (LSTM) networks in classifying emotions from continuous data.

Main Methods:

• A thresholding scheme was developed to convert continuous emotional traces into three distinct emotional states over time.
• This scheme was applied to a long short-term memory (LSTM) network framework for emotion classification.
• Electroencephalography (EEG) signals and frontal facial video data from the MAHNOB-HCI dataset were utilized for feature extraction.

Main Results:

• The proposed thresholding scheme significantly improved the three-class classification accuracy of the EEG feature-based LSTM network.
• A statistically significant improvement was observed with a p-value of 0.0329.
• The temporal annotation approach demonstrated superior performance compared to traditional fixed annotation methods.

Conclusions:

• The developed thresholding scheme effectively addresses the limitations of fixed annotation in emotion classification.
• Temporally annotating continuous emotional data enhances the accuracy of LSTM-based emotion recognition systems.
• This approach offers a more nuanced and accurate method for analyzing dynamic emotional states using physiological signals like EEG.