Improving Generalized Visual Grounding With Instance-Aware Joint Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Generalized visual grounding extends classical methods to multi-target and non-target scenarios.
• Existing Generalized Referring Expression Comprehension (GREC) and Segmentation (GRES) approaches are often trained independently.
• Current Generalized Visual Grounding Segmentation (GRES) methods overlook instance-aware capabilities and box-mask consistency.

Purpose of the Study:

• To propose InstanceVG, a unified multi-task framework for generalized visual grounding.
• To address limitations in independent GREC and GRES training and incorporate instance-aware capabilities.
• To ensure consistent multi-granularity predictions at both bounding box and pixel levels.

Main Methods:

• Developed InstanceVG, a novel multi-task framework for joint GREC and GRES.
• Incorporated instance queries to unify joint and consistent predictions of instance-level boxes and masks.
• Utilized prior reference points for each instance query to enhance target matching and consistency.

Main Results:

• InstanceVG achieves state-of-the-art performance across ten datasets and four tasks.
• Demonstrated significant improvements over existing methods in various evaluation metrics.
• Validated the effectiveness of instance-aware capabilities for generalized visual grounding.

Conclusions:

• InstanceVG is the first framework to simultaneously address GREC and GRES with instance-aware capabilities.
• The proposed approach ensures consistent predictions across different granularities (points, boxes, masks).
• InstanceVG offers a streamlined and more effective solution for generalized visual grounding tasks.