Constrained Structure Learning for Scene Graph Generation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Scene graph generation is a structured prediction task crucial for visually-grounded scene understanding.
• Current methods predominantly use the mean field variational Bayesian framework with message passing neural networks for inference.
• Existing approaches overlook the potential of constrained optimization models for scene graph generation.

Purpose of the Study:

• To propose a novel constrained structure learning method for scene graph generation.
• To introduce an explicit constrained variational inference objective.
• To explore alternative inference strategies beyond message passing.

Main Methods:

• Developed a constrained structure learning framework for scene graph generation.
• Proposed an explicit constrained variational inference objective.
• Employed entropic mirror descent, a generic constrained optimization method, for the inference step, replacing traditional message passing.

Main Results:

• The proposed constrained structure learning method was validated on popular scene graph generation benchmarks.
• The generic constrained optimization approach demonstrated superior performance compared to existing state-of-the-art methods.
• The study successfully explored constrained optimization models for scene graph generation.

Conclusions:

• The novel constrained structure learning method offers a more effective approach to scene graph generation.
• Entropic mirror descent provides a viable and high-performing alternative for constrained variational inference in this task.
• The findings suggest a promising direction for future research in structured prediction and scene understanding.