Importance Weighted Structure Learning for Scene Graph Generation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Scene graph generation models objects and relationships in images.
• Current methods use message passing neural networks and evidence lower bound, which can underestimate posterior distributions.
• This leads to suboptimal performance in generating accurate scene graphs.

Purpose of the Study:

• To propose a novel importance weighted structure learning method for scene graph generation.
• To address the underestimation of posterior distributions in existing approaches.
• To improve the performance and accuracy of visual scene graph generation.

Main Methods:

• Developed an importance weighted structure learning approach.
• Utilized a reparameterizable Gumbel-Softmax sampler to draw multiple samples.
• Applied an entropic mirror descent algorithm for constrained variational inference.
• Approximated the log-partition function using a tighter importance weighted lower bound.

Main Results:

• The proposed method achieves state-of-the-art performance on popular scene graph generation benchmarks.
• Demonstrated superior accuracy in modeling objects and their relationships.
• Showcased the effectiveness of the tighter lower bound approximation.

Conclusions:

• The novel importance weighted structure learning method significantly enhances scene graph generation.
• The approach overcomes limitations of traditional evidence lower bound methods.
• This work advances the field of structured prediction for visual understanding.