Debiased Scene Graph Generation for Dual Imbalance Learning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene graph generation (SGG) is vital for understanding visual semantics but faces significant challenges due to dual data imbalance.
• Existing SGG methods often neglect the background-foreground imbalance, focusing solely on foreground relationship skewness, leading to biased models.

Purpose of the Study:

• To propose a novel debiased SGG method (DSDI) that addresses both background-foreground and foreground relationship imbalances.
• To develop techniques that mitigate bias in SGG models for broader applicability in downstream tasks.

Main Methods:

• Introduced a novel debiased SGG method (DSDI) incorporating biased resistance loss and a causal intervention tree.
• Biased resistance loss decouples background classification from foreground relationship recognition to enhance foreground feature representation.
• A causal graph and intervention tree were employed to eliminate context bias and learn unbiased relationship features.

Main Results:

• DSDI demonstrated superior performance over state-of-the-art methods on two highly imbalanced datasets (VG150 and VrR-VG).
• The proposed method effectively reduces the side effects of dual imbalance, enabling more equitable contributions from different categories.

Conclusions:

• The DSDI method offers a significant advancement in addressing dual data imbalance in scene graph generation.
• This approach enhances model robustness and applicability for real-world computer vision applications.