SGTR+: End-to-End Scene Graph Generation With Transformer.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene Graph Generation (SGG) is a complex visual understanding task due to its compositional nature.
• Existing SGG methods often face high time complexity or suboptimal design choices.
• Current approaches include bottom-up, two-stage, or point-based, one-stage methods.

Purpose of the Study:

• To propose a novel and efficient Scene Graph Generation (SGG) method.
• To address the limitations of existing SGG approaches, such as high time complexity.
• To formulate SGG as a bipartite graph construction problem.

Main Methods:

• A transformer-based, end-to-end framework is developed for SGG.
• The framework generates entity and entity-aware predicate proposal sets.
• A graph assembling module infers connectivity for bipartite scene graph construction.

Main Results:

• The proposed method achieves optimal performance and time-complexity.
• State-of-the-art or comparable results were obtained on three challenging benchmarks.
• The approach demonstrates higher inference efficiency compared to existing methods.

Conclusions:

• The bipartite graph assembling paradigm offers an effective approach to SGG.
• Enhanced entity-aware modeling and stable graph assembling contribute to optimal performance.
• The novel framework provides a more efficient and effective solution for Scene Graph Generation.