Word2Pix: Word to Pixel Cross-Attention Transformer in Visual Grounding

Area of Science:

• Computer Vision
• Artificial Intelligence
• Natural Language Processing

Background:

• Current one-stage visual grounding methods use holistic sentence embeddings, limiting word-level query understanding.
• This can lead to neglecting critical words for accurate object identification.

Purpose of the Study:

• To propose Word2Pix, a one-stage visual grounding network.
• To enable word-level textual-visual feature correspondence for improved target localization.

Main Methods:

• Utilizes an encoder-decoder transformer architecture.
• Employs word-to-pixel attention via multiple transformer decoder layers.
• Learns simultaneous language modeling, feature fusion, and target prediction.

Main Results:

• Word2Pix outperforms existing one-stage visual grounding methods on RefCOCO, RefCOCO+, and RefCOCOg datasets.
• Achieves superior performance compared to two-stage visual grounding models.
• Maintains end-to-end training and fast inference speeds characteristic of one-stage methods.

Conclusions:

• Word2Pix offers a significant advancement in one-stage visual grounding.
• The word-to-pixel attention mechanism effectively addresses limitations of holistic sentence embeddings.
• Provides a faster and more accurate alternative to existing visual grounding techniques.