Cycle-Consistent Weakly Supervised Visual Grounding With Individual and Contextual Representations.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Visual grounding connects image regions with text, crucial for cross-modal learning.
• Weakly supervised visual grounding lacks fine-grained correspondence, leading to matching ambiguity.

Purpose of the Study:

• To address matching ambiguity in weakly supervised visual grounding.
• To improve the alignment accuracy between image regions and textual queries.

Main Methods:

• Introduced a cycle consistency constraint for region-phrase pairs to strengthen correct matches and weaken incorrect ones.
• Proposed a parallel grounding framework extracting individual and contextual representations.
• Calculated context-free and context-aware similarities for regions and phrases.
• Trained the framework using image-text contrastive loss and cycle consistency loss.

Main Results:

• Achieved noticeable improvement in visual grounding performance across five benchmark datasets.
• Demonstrated the effectiveness of cycle consistency in alleviating matching ambiguity.
• Showcased the complementary nature of individual and contextual representations for cross-modal alignment.

Conclusions:

• The proposed method effectively enhances weakly supervised visual grounding.
• Cycle consistency and the parallel framework offer a robust solution for cross-modal alignment.
• The approach provides a significant step forward in accurately grounding text in images under weak supervision.