Progressive Language-Customized Visual Feature Learning for One-Stage Visual Grounding.

Area of Science:

• Computer Vision
• Natural Language Processing
• Artificial Intelligence

Background:

• Visual grounding localizes sentence-described objects in images.
• Current methods often separate visual and linguistic feature extraction, limiting cross-modal interaction.
• This separation may underutilize information from both modalities.

Purpose of the Study:

• To propose a novel language-customized visual feature learning mechanism.
• To enhance cross-modal interaction during feature extraction for visual grounding.
• To develop a more effective and efficient visual grounding framework.

Main Methods:

• Introduced Progressive Language-customized Visual feature learning (PLV), a one-stage framework.
• Developed a Progressive Language-customized Visual Encoder (PLVE) guided by linguistic information.
• Utilized Channel-wise Language-guided Interaction Modules (CLIM) for stage-wise visual feature customization.

Main Results:

• Achieved state-of-the-art performance on five visual grounding datasets.
• Demonstrated significant performance improvements over conventional methods.
• Realized real-time processing speeds without pre-training on object detection datasets.

Conclusions:

• The proposed PLV framework effectively integrates linguistic guidance for improved visual grounding.
• Early cross-modal interaction during feature extraction is crucial for optimal performance.
• PLV offers a promising, efficient, and accurate solution for visual grounding tasks.