Human-Inspired Scene Understanding: A Grounded Cognition Method for Unbiased Scene Graph Generation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene Graph Generation (SGG) is crucial for image understanding but suffers from long-tailed bias, favoring common visual relationships.
• Existing unbiased SGG methods struggle with dataset variations due to limited generalization.
• Explicitly modeling class diversity is essential for robust SGG.

Purpose of the Study:

• To propose a novel Grounded Cognition Method (GCM) for unbiased Scene Graph Generation.
• To enhance the generalization ability of SGG models by incorporating human-like cognitive processes.
• To mitigate the long-tailed bias prevalent in current SGG approaches.

Main Methods:

• GCM models simulations via Out Domain Knowledge Injection and Semantic Group Aware Synthesizer.
• Bodily states are simulated by modality erasure to improve cross-modal compensation.
• Situated actions are modeled using a Shapley Enhanced Multimodal Counterfactual module for contextual understanding.

Main Results:

• GCM significantly outperforms state-of-the-art methods on Visual Genome, GQA, and Open Images V6 datasets.
• The proposed method effectively alleviates long-tailed bias in scene graph generation.
• GCM achieves a balanced performance between head (common) and tail (rare) visual relationships.

Conclusions:

• Grounded Cognition Method offers a robust solution for unbiased Scene Graph Generation.
• The approach enhances model generalization and addresses limitations of existing methods.
• GCM provides a balanced performance, improving understanding of both common and rare visual contexts.