TransVG++: End-to-End Visual Grounding With Language Conditioned Vision Transformer.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Traditional visual grounding methods rely on complex, manually-designed fusion mechanisms.
• These heuristic designs can lead to overfitting and suboptimal performance on diverse datasets.

Purpose of the Study:

• To develop simplified yet effective Transformer-based frameworks for visual grounding.
• To improve multi-modal fusion and reasoning capabilities in visual grounding models.
• To establish new state-of-the-art performance on visual grounding tasks.

Main Methods:

• Proposed TransVG, using Transformers for multi-modal correspondence and direct box coordinate regression.
• Introduced TransVG++, a purely Transformer-based framework leveraging Vision Transformer (ViT).
• Developed Language Conditioned Vision Transformer for intermediate vision-language fusion, reusing ViT components.

Main Results:

• Demonstrated that simple Transformer encoder layers outperform complex fusion modules in TransVG.
• TransVG++ achieved significant performance gains by integrating ViT and intermediate fusion.
• Reported state-of-the-art records across five prevalent visual grounding datasets.

Conclusions:

• Transformer-based approaches offer a more effective and less complex alternative to traditional visual grounding methods.
• The TransVG++ framework, with its integrated Vision Transformer, represents a significant advancement in visual grounding accuracy and efficiency.
• The proposed methods pave the way for more robust and generalizable visual grounding systems.