Context Disentangling and Prototype Inheriting for Robust Visual Grounding.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Visual grounding (VG) identifies image targets from language queries, but struggles with context discrimination and open-vocabulary generalization.
• Existing methods often overlook contextual information, limiting performance when targets share categories or appear in novel scenes.

Purpose of the Study:

• To develop a robust visual grounding framework capable of handling both standard and open-vocabulary scenes.
• To improve the discrimination of target objects by effectively utilizing contextual information.
• To enhance generalization to novel objects and categories not seen during training.

Main Methods:

• Proposing a novel framework featuring context disentangling and prototype inheriting for visual grounding.
• Context disentangling separates referent and context features for improved discrimination.
• Prototype inheriting utilizes a prototype bank to leverage seen data, particularly for open-vocabulary scenarios.
• Fused features from disentangled linguistic and visual prototypes are processed by a Vision Transformer for bounding box regression.

Main Results:

• The proposed method demonstrates superior performance in distinguishing targets, even those of the same category.
• Context disentangling effectively enhances the discriminative power of visual features.
• Prototype inheriting significantly improves performance in open-vocabulary visual grounding tasks.
• Extensive experiments confirm the method's outperformance against state-of-the-art approaches on both standard and open-vocabulary scenes.

Conclusions:

• The novel framework offers a robust solution for visual grounding across diverse scenes, including those with novel objects.
• Context disentangling and prototype inheriting are key components for achieving high performance and generalization.
• The approach advances the state-of-the-art in visual grounding, particularly for challenging open-vocabulary scenarios.