Confidence-Aware Pseudo-Label Self-Correction for Weakly Supervised Visual Grounding

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Weakly supervised visual grounding aims to connect text queries to image regions without direct training data.
• Existing methods struggle with overfitting due to unreliable cross-modal similarity scores for region proposal selection.

Purpose of the Study:

• To develop a robust framework for weakly supervised visual grounding that overcomes model overfitting.
• To improve the accuracy and reliability of associating text queries with image regions.

Main Methods:

• Proposed Confidence-aware Pseudo-label Learning (CPL) framework using uni-modal similarity for reliable pseudo-label generation.
• Introduced a cross-modal verification module using pre-trained vision-language models.
• Developed CPL++ with dynamic verification based on grounding loss and a self-supervised association correction module.

Main Results:

• Experimental results on five datasets demonstrate the superiority of the proposed approach.
• The CPL++ framework effectively mitigates error propagation by dynamically verifying and correcting suspicious associations.
• The methods show improved performance in weakly supervised visual grounding tasks.

Conclusions:

• The proposed CPL and CPL++ frameworks offer significant improvements in weakly supervised visual grounding.
• Dynamic verification and self-supervised correction are effective strategies for handling unreliable associations.
• The approach advances the state-of-the-art in visual grounding by addressing model overfitting and error propagation.