Tensor-Based Emotional Category Classification via Visual Attention-Based Heterogeneous CNN Feature Fusion.

Area of Science:

• Computer Vision
• Affective Computing
• Machine Learning

Background:

• Human emotions are intrinsically linked to visual attention patterns.
• Analyzing eye gaze provides insights into cognitive and emotional states.
• Existing emotion classification methods often lack robust feature representation for dynamic visual attention.

Purpose of the Study:

• To propose a novel method for emotion classification leveraging visual attention and eye gaze analysis.
• To develop a tensor-based approach for fusing heterogeneous convolutional neural network (CNN) features.
• To enhance the accuracy and representation capabilities of emotion classification models.

Main Methods:

• Gaze-based image representation to capture temporal changes in visual attention.
• Extraction of multiple CNN features from the gaze-based representation.
• Fusion of CNN features using a novel tensor construction for visual attention-based heterogeneous CNN feature fusion.
• Application of logistic tensor regression and general tensor discriminant analysis for classification.

Main Results:

• The proposed method achieved an F1-measure of approximately 0.6 for emotion classification.
• Demonstrated a significant improvement of about 10% compared to state-of-the-art methods.
• Effectiveness of the tensor-based feature fusion for visual attention-based emotion recognition was verified.

Conclusions:

• The developed method offers a promising approach for accurate emotion classification through eye gaze analysis.
• Tensor-based fusion of CNN features effectively captures visual attention dynamics for improved emotion recognition.
• The findings highlight the potential of integrating eye-tracking data with deep learning for affective computing applications.