Seeking Subjectivity in Visual Emotion Distribution Learning.

Area of Science:

• Computer Vision
• Affective Computing
• Machine Learning

Background:

• Visual Emotion Analysis (VEA) aims to predict emotions evoked by visual stimuli.
• VEA is often treated as Label Distribution Learning (LDL) due to individual voting variations.
• Current methods overlook the subjective nature of crowd-sourced emotion data.

Purpose of the Study:

• To propose a novel Subjectivity Appraise-and-Match Network (SAMNet) for VEA.
• To model the inherent subjectivity in visual emotion distribution.
• To improve the accuracy and interpretability of emotion prediction models.

Main Methods:

• Developed a multi-branch Subjectivity Appraising module to simulate individual emotion evocation.
• Incorporated an attention-based mechanism to construct affective memory for preserving unique emotional experiences.
• Introduced a subjectivity loss to ensure divergence between individual predictions and a matching loss with the Hungarian algorithm for label assignment.

Main Results:

• SAMNet demonstrated superior performance compared to state-of-the-art methods on public visual emotion distribution datasets.
• Ablation studies confirmed the effectiveness of the proposed components.
• Visualizations highlighted the interpretability of the SAMNet model.

Conclusions:

• The proposed SAMNet effectively captures and models subjectivity in visual emotion analysis.
• The approach enhances the prediction of emotion distributions by considering individual appraisal processes.
• SAMNet offers a more nuanced and interpretable solution for VEA tasks.