Discriminative Triad Matching and Reconstruction for Weakly Referring Expression Grounding

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly-supervised referring expression grounding aims to localize objects in images based on natural language descriptions.
• Existing methods struggle with accurate region-query matching due to the absence of explicit training data.
• Traditional approaches use region-query matching and sentence reconstruction, which are approximations that can lead to errors.

Purpose of the Study:

• To develop a novel and effective method for weakly-supervised referring expression grounding.
• To improve the accuracy and efficiency of object localization in images using textual queries.
• To address the limitations of existing methods in handling the unknown correctness of matching.

Main Methods:

• Proposes a discriminative triad approach to convert queries into scalable discriminative triads.
• Introduces triad-level matching and reconstruction modules for efficient weakly-supervised training.
• The method is designed to be lightweight and faster than previous state-of-the-art techniques.

Main Results:

• Achieves new state-of-the-art accuracy on RefCOCO (39.21%), RefCOCO+ (39.18%), and RefCOCOg (43.24%) datasets.
• Demonstrates significant performance improvements of 4.17%, 4.08%, and 7.8% over prior methods, respectively.
• The proposed method is three times lighter and faster than existing approaches.

Conclusions:

• The discriminative triad approach offers a simple yet highly effective solution for weakly-supervised referring expression grounding.
• The method achieves superior performance and efficiency, setting a new benchmark in the field.
• Code availability facilitates further research and development in this area.