Predicate Correlation Learning for Scene Graph Generation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Scene Graph Generation (SGG) methods often exhibit performance disparities between common (head) and rare (tail) predicate classes.
• This performance gap is attributed to semantic overlap among predicates and the inherent long-tailed distribution of visual data.

Purpose of the Study:

• To propose a novel Predicate Correlation Learning (PCL) method to mitigate the head-tail performance gap in SGG.
• To enhance the accuracy of recognizing rare predicate classes by leveraging inter-predicate relationships.

Main Methods:

• Introduced a Predicate Correlation Matrix (PCM) to quantify semantic relationships between predicate pairs.
• Dynamically updated the PCM to mitigate long-tailed bias within the predicate correlation matrix.
• Integrated the PCM into a predicate correlation loss function (LPC) to improve gradient flow for underrepresented classes.

Main Results:

• The PCL method significantly improved the performance of tail classes in Scene Graph Generation.
• Evaluations on multiple benchmarks demonstrated substantial gains when PCL was built upon existing SGG frameworks.
• Effectively reduced the negative impact of semantic overlap and data imbalance on predicate recognition.

Conclusions:

• Predicate Correlation Learning (PCL) is an effective approach to address performance disparities in SGG.
• Considering predicate correlations offers a promising direction for improving the robustness and accuracy of visual understanding models.
• The proposed PCM and LPC contribute to more balanced and accurate scene graph generation, particularly for tail predicates.