A Real-Time Global Inference Network for One-Stage Referring Expression Comprehension.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Referring Expression Comprehension (REC) is crucial for image understanding.
• Existing REC methods often use computationally expensive multistage pipelines, limiting real-time applications.
• Addressing expression diversity and complexity is key for robust REC.

Purpose of the Study:

• To propose a novel one-stage model for real-time Referring Expression Comprehension (REC).
• To enhance REC performance by tackling expression diversity and complexity.
• To achieve significant speed improvements over existing REC methods.

Main Methods:

• Introduced the real-time global inference network (RealGIN), a one-stage REC model.
• Developed Adaptive Feature Selection (AFS) to fuse multi-level semantic features, addressing expression diversity.
• Implemented Global Attentive ReAsoNing (GARAN) to model complex relational conditions using text as a pivot.

Main Results:

• RealGIN demonstrated superior performance on five benchmark datasets (RefCOCO, RefCOCO+, RefCOCOg, ReferIT, Flickr30k).
• Achieved highly competitive results against state-of-the-art methods like MAttNet.
• RealGIN achieved a 10-20x processing speed increase compared to existing methods on identical hardware.

Conclusions:

• The proposed RealGIN model offers an efficient and effective solution for real-time referring expression comprehension.
• AFS and GARAN are innovative components that successfully handle expression diversity and complexity.
• RealGIN significantly advances the state-of-the-art in real-time REC, enabling broader applications.