SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network.

Area of Science:

• Computer Science
• Artificial Intelligence
• Psychology

Background:

• Visual Emotion Analysis (VEA) is gaining attention due to social media image sharing.
• Inferring emotions from images is challenging due to the complex cognitive processes involved.
• Psychological studies show emotions arise from interactions between objects and scenes.

Purpose of the Study:

• To propose a novel network, SOLVER, for improved visual emotion prediction.
• To model the intricate relationships between objects and scenes for emotion inference.
• To enhance the accuracy and interpretability of visual emotion analysis.

Main Methods:

• Constructed an Emotion Graph using semantic and visual features to capture object relationships.
• Employed Graph Convolutional Networks (GCN) for reasoning on the Emotion Graph, generating emotion-enhanced object features.
• Developed a Scene-Object Fusion Module with a scene-based attention mechanism to integrate scene and object information.

Main Results:

• SOLVER significantly outperformed state-of-the-art methods on eight public visual emotion datasets.
• Ablation studies confirmed the effectiveness of individual components of the SOLVER method.
• Visualizations demonstrated the interpretability of the model, offering new insights into VEA.

Conclusions:

• The proposed SOLVER network effectively predicts emotions from images by modeling scene-object interrelations.
• SOLVER demonstrates robustness across multiple datasets, though some limitations were identified.
• This work advances visual emotion analysis by providing a more accurate and interpretable approach.