Contextual Transformer Networks for Visual Recognition.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Transformers with self-attention have revolutionized Natural Language Processing and are increasingly applied to Computer Vision.
• Existing Transformer designs often overlook contextual information among neighboring keys in feature maps.
• This limitation hinders the full potential of self-attention for robust visual representation.

Purpose of the Study:

• To introduce a novel Transformer-style module, the Contextual Transformer (CoT) block, for visual recognition tasks.
• To enhance visual representation capacity by exploiting contextual information among keys.
• To develop a stronger Transformer-style backbone for various computer vision applications.

Main Methods:

• The Contextual Transformer (CoT) block is proposed, incorporating contextual information of keys.
• Keys are contextually encoded using a 3x3 convolution, followed by concatenation with queries.
• A dynamic multi-head attention matrix is learned via 1x1 convolutions, fused with static representations.

Main Results:

• The CoT block can replace standard 3x3 convolutions in ResNet architectures, forming Contextual Transformer Networks (CoTNet).
• CoTNet demonstrates superior performance as a backbone across diverse applications including image recognition, object detection, and segmentation.
• Extensive experiments validate the effectiveness of the proposed contextual approach.

Conclusions:

• The Contextual Transformer (CoT) block effectively utilizes contextual information to enhance visual representations.
• CoTNet offers a powerful and versatile Transformer-style backbone for computer vision.
• The proposed method represents a significant advancement in leveraging attention mechanisms for visual tasks.