MLMG-SGG: Multilabel Scene Graph Generation With Multigrained Features.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Scene graph generation (SGG) is crucial for image understanding but typically assumes single relationships between object pairs.
• Existing SGG methods often fail to capture the complexity of multiple semantic relationships co-existing between objects.

Purpose of the Study:

• To develop a novel multi-label scene graph generation (SGG) pipeline capable of identifying multiple relationships between object pairs.
• To enhance SGG by incorporating multi-grained features for a more comprehensive scene representation.

Main Methods:

• Proposed a multi-label scene graph generation pipeline (MLMG-SGG) that treats relationship detection as a multi-label classification problem.
• Introduced a Multi-Grained Module (MGM) to encode object features at different spatial scales (object-level and region-level).
• Generated multi-graphs at inference time to represent multiple relationships.

Main Results:

• The MLMG-SGG pipeline demonstrated significant performance gains on a benchmark dataset.
• The proposed method effectively models fine-grained relationships by utilizing multi-grained features.
• The pipeline acts as a plug-in, enhancing state-of-the-art SGG methods.

Conclusions:

• The multi-label approach with multi-grained features significantly advances scene graph generation.
• MLMG-SGG provides a more accurate and detailed representation of semantic relationships in images.
• This method offers a flexible and effective solution for complex scene understanding tasks.