Unbiased Scene Graph Generation via Two-Stage Causal Modeling.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Existing Scene Graph Generation (SGG) methods primarily address long-tailed distribution bias.
• Semantic confusion, leading to incorrect predictions for similar relationships, remains an overlooked bias in SGG.
• Noisy datasets introduce unobserved confounders, limiting the effectiveness of standard causal inference techniques like Sparse Mechanism Shift (SMS).

Purpose of the Study:

• To develop a novel debiasing procedure for Scene Graph Generation (SGG) leveraging causal inference.
• To address both long-tailed distribution and semantic confusion biases simultaneously.
• To propose a method that preserves performance on common (head) categories while improving predictions for rare (tail) relationships.

Main Methods:

• Proposed Two-stage Causal Modeling (TsCM) to handle multiple biases in SGG.
• Implemented a causal representation learning stage using Population Loss (P-Loss) to mitigate semantic confusion.
• Introduced Adaptive Logit Adjustment (AL-Adjustment) in a second stage for causal calibration learning against long-tailed distribution bias.
• Ensured model-agnostic nature of TsCM for compatibility with various SGG architectures.

Main Results:

• TsCM achieved state-of-the-art performance on popular SGG benchmarks, measured by mean recall rate.
• The method demonstrated superior recall rates compared to existing debiasing techniques.
• TsCM effectively balanced performance between head and tail relationships, indicating a better tradeoff.

Conclusions:

• The proposed Two-stage Causal Modeling (TsCM) effectively addresses semantic confusion and long-tailed distribution biases in SGG.
• TsCM offers a robust and adaptable debiasing solution for SGG models.
• The findings suggest a promising direction for developing more accurate and unbiased scene graph generation systems.