Learning to Compose and Reason with Language Tree Structures for Visual Grounding.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Image grounding requires understanding fine-grained language compositions.
• Existing models often use holistic features, neglecting composite language reasoning.
• Accurate localization of objects described by complex phrases is challenging.

Purpose of the Study:

• To develop a natural language grounding model that addresses the limitations of holistic feature association.
• To enable visual reasoning based on the compositional structure of language.
• To improve the explainability of AI models in image-text tasks.

Main Methods:

• Proposed RvG-Tree (Recursive Grounding Tree) model for parsing language into a binary tree structure.
• Implemented a bottom-up visual reasoning approach along the constructed tree.
• Utilized the Straight-Through Gumbel-Softmax estimator for end-to-end training of the discrete tree construction.

Main Results:

• Achieved state-of-the-art performance on multiple benchmark datasets for natural language grounding.
• Demonstrated improved explainability in the model's visual reasoning process.
• RvG-Tree effectively handles complex language compositions through recursive decomposition.

Conclusions:

• RvG-Tree offers a more effective approach to natural language grounding by leveraging compositional language structure.
• The model's recursive, bottom-up reasoning provides explainable insights into the grounding process.
• This method advances the field of AI by enabling more nuanced comprehension of language in visual contexts.