A Causal Adjustment Module for Debiasing Scene Graph Generation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Existing Scene Graph Generation (SGG) debiasing methods primarily focus on relationship distribution, neglecting underlying object and object pair distribution biases.
• This oversight limits the effectiveness of current debiasing strategies in SGG.

Purpose of the Study:

• To investigate the root causes of bias in Scene Graph Generation (SGG) using causal inference.
• To develop a novel method for debiasing SGG models by addressing skewed object and object pair distributions.

Main Methods:

• Employed causal inference techniques to model the causal relationships between object, object pair, and relationship distributions.
• Introduced the Mediator-based Causal Chain Model (MCCM) incorporating mediator variables like co-occurrence distributions.
• Proposed the Causal Adjustment Module (CAModule) to estimate causal structures and generate adjustment factors for bias correction.

Main Results:

• The proposed Causal Adjustment Module (CAModule) significantly improves state-of-the-art mean recall rates in SGG.
• Demonstrated substantial enhancements in zero-shot relationship recall rates.
• Validated effectiveness across various SGG backbones and benchmarks.

Conclusions:

• Skewed object and object pair distributions are profound causes of bias in SGG, beyond relationship distribution.
• The Mediator-based Causal Chain Model (MCCM) and Causal Adjustment Module (CAModule) offer an effective causal inference-based approach to debias SGG.
• The proposed method enhances SGG performance and enables zero-shot relationship composition.