Visual Grounding Via Accumulated Attention.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Visual grounding (VG) aims to connect natural language queries to specific image regions.
• Real-world applications face challenges with ambiguous queries and complex scenes, leading to performance limitations.
• Existing methods struggle with redundant and correlated information in images.

Purpose of the Study:

• To enhance the accuracy and robustness of visual grounding models.
• To address challenges posed by ambiguous queries and complex image structures.
• To improve the localization of target objects based on natural language descriptions.

Main Methods:

• Exploiting attention modules to reduce internal redundancies within different information types.
• Proposing an accumulated attention (A-ATT) mechanism for joint reasoning across attention modules.
• Implementing a "noised" training strategy to bridge the gap between training data and real-world data quality.
• Learning a bounding box regressor for refining target object localization.

Main Results:

• The proposed A-ATT mechanism explicitly captures relations among different information types.
• The "noised" training strategy improves model robustness and performance on real-world data.
• Experimental results on four benchmark datasets demonstrate significant outperformance over previous works.

Conclusions:

• The developed methods significantly advance the state-of-the-art in visual grounding.
• The A-ATT mechanism and "noised" training strategy are effective in handling complex visual grounding tasks.
• The approach shows strong potential for real-world applications requiring accurate object localization from language queries.