Bridging Subjectivity in Affective Explanation Captioning via Consensus-Prompted Emotion Reasoning

Area of Science:

• Computer Vision
• Artificial Intelligence
• Cognitive Science

Background:

• Affective Explanation Captioning (AEC) aims to analyze viewer-centered emotions in images and explain their causes.
• Existing methods struggle with emotional subjectivity and the gap between abstract emotions and visual content.

Purpose of the Study:

• To propose a novel framework, Consensus-Prompted Emotion Reasoning (CPER), to address AEC challenges.
• To model emotional diversity and enforce emotional-semantic alignment for improved visual emotion analysis.

Main Methods:

• Developed a consensus-based bucket prompt to manage emotional diversity.
• Introduced a dual-space representation using CLIP and emotion encoders for semantic and affective cues.
• Implemented an emotion consistency learning strategy for grounded explanations.

Main Results:

• CPER demonstrated effectiveness in handling emotional diversity and semantic coherence.
• Experiments on ArtEmis v1.0, ArtEmis v2.0, and Affection datasets showed superior performance.
• The framework achieved better results compared to state-of-the-art methods in AEC.

Conclusions:

• CPER successfully bridges the gap between subjective emotions and objective visual content.
• The proposed methods enhance the accuracy and coherence of affective explanation captions.
• CPER offers a promising approach for viewer-centered visual emotion analysis.