Global Context Networks.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Non-local network (NLNet) effectively captures long-range dependencies in images by aggregating query-specific global context.
• Empirical analysis revealed that NLNet's global contexts are largely similar across different query positions.
• This redundancy presents an opportunity for network simplification and computational efficiency.

Purpose of the Study:

• To develop a simplified network that maintains NLNet's accuracy with reduced computational cost.
• To introduce a query-independent formulation for modeling global context more efficiently.
• To enhance the efficiency of modeling global context in deep neural networks for image recognition.

Main Methods:

• Developed a simplified network based on a query-independent formulation, leveraging the finding that global contexts are similar across positions.
• Replaced the single-layer transformation in non-local blocks with a two-layer bottleneck structure.
• Introduced the global context (GC) block, a lightweight module for modeling global context, integrated into a global context network (GCNet).

Main Results:

• The proposed global context (GC) block significantly reduces computation and parameter count compared to standard non-local blocks.
• GCNet, utilizing the GC block, maintains the accuracy of NLNet across various image recognition tasks.
• Experiments demonstrate that GCNet generally outperforms NLNet on major benchmarks.

Conclusions:

• The query-independent formulation of global context is an effective and efficient alternative to query-specific aggregation.
• The lightweight GC block can be readily integrated into backbone networks, offering substantial computational savings.
• GCNet provides a more efficient yet equally accurate approach for capturing long-range dependencies in image recognition tasks.