RepSGG: Novel Representations of Entities and Relationships for Scene Graph Generation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene Graph Generation (SGG) traditionally uses fixed-size entity representations, leading to performance and computational trade-offs.
• Efficient and dynamic feature extraction is crucial for advancing SGG capabilities.

Purpose of the Study:

• To introduce a novel architecture, RepSGG, for more effective and efficient Scene Graph Generation.
• To address the challenges of fixed-size representations and the long-tailed distribution in SGG.

Main Methods:

• Proposed RepSGG architecture: subjects as queries, objects as keys, and relationships as maximum attention weights.
• Developed a performance-guided logit adjustment (PGLA) strategy to balance dominant and rare relationship classes during training.

Main Results:

• RepSGG demonstrates superior fine-grained and flexible representation power for entities and relationships.
• Achieved state-of-the-art or comparable performance on Visual Genome and Open Images V6 datasets.
• Exhibited fast inference speed, highlighting the method's efficiency.

Conclusions:

• RepSGG offers an effective and efficient approach to Scene Graph Generation.
• The dynamic representation and PGLA strategy significantly improve SGG performance and generalization, especially for long-tailed distributions.