Entity-Enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Natural Language Processing

Background:

• Weakly supervised referring expression grounding (REG) lacks explicit target-expression correspondence.
• Existing methods struggle with proposal-query ambiguity and distinguishing targets from similar objects due to ignored location and context.

Purpose of the Study:

• To address the limitations of current weakly supervised REG methods.
• To improve the accuracy and robustness of grounding language expressions to image targets.

Main Methods:

• Proposed the entity-enhanced adaptive reconstruction network (EARN).
• EARN comprises three modules: entity enhancement for proposal selection, adaptive grounding using hierarchical attention for ranking, and collaborative reconstruction for result measurement.
• Employs semantic similarity, subject-location-context ranking, and adaptive/language reconstruction with attribute classification.

Main Results:

• EARN significantly outperforms state-of-the-art methods across five datasets.
• Qualitative results show improved performance in scenarios with multiple same-category objects.

Conclusions:

• The proposed EARN effectively handles ambiguities in weakly supervised REG.
• EARN demonstrates superior performance by integrating entity enhancement, adaptive grounding, and collaborative reconstruction, particularly in complex scenes.