State-Aware Compositional Learning Toward Unbiased Training for Scene Graph Generation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene graph generation (SGG) models often exhibit prediction biases.
• Current debiasing methods like resampling and causality analysis have limitations.
• The influence of intrinsic features on SGG bias remains under-explored.

Purpose of the Study:

• To identify the primary cause of biased predictions in SGG.
• To propose a novel method for decoupling features and mitigating bias.
• To address data scarcity issues for rare relations in SGG.

Main Methods:

• Empirically demonstrate that object label embeddings alone can lead to biased SGG predictions.
• Develop a conditional variational auto-encoder to separate intrinsic identity and extrinsic relation-dependent features.
• Introduce compositional learning strategies for both relation and object levels to handle rare relations.

Main Results:

• Object identity information, specifically label embeddings, is identified as the principal source of bias in SGG.
• The proposed Decomposition and Composition (DeC) method effectively mitigates bias in relation prediction.
• DeC improves the performance of existing SGG models across conventional, few-shot, and zero-shot scenarios.

Conclusions:

• Decoupling features using DeC is a promising approach to address bias in SGG.
• The DeC method offers a model-agnostic solution that enhances SGG performance.
• This research opens new avenues for developing fairer and more accurate SGG models.