RA-Net: reverse attention for generalizing residual learning.

Area of Science:

• Computer Vision
• Deep Learning
• Neural Network Architectures

Background:

• Residual learning utilizes identity mapping for unimpeded information transfer in deep networks.
• While beneficial, unhindered transmission can negatively impact network performance.
• Existing methods lack mechanisms to selectively filter or guide information flow within identity mappings.

Purpose of the Study:

• To introduce a generalized residual learning architecture, Reverse Attention (RA), to improve information transmission in deep neural networks.
• To address performance degradation caused by interference in standard residual learning.
• To enhance the effectiveness of identity mapping through semantic feature guidance.

Main Methods:

• Proposed Reverse Attention (RA) architecture, applying high-level semantic features to supervise low-level information in the identity mapping branch.
• Introduced Modified Global Response Normalization (M-GRN) to implement the reverse attention mechanism.
• Integrated M-GRN into the residual learning framework to create RA-Net.

Main Results:

• RA-Net demonstrated significant improvements over standard residual networks on various computer vision tasks.
• Achieved a 1.7% increase in Top-1 accuracy on ImageNet-1K classification compared to ResNet101, with similar parameters and computational cost.
• Improved box Average Precision (AP) by 1.9% on COCO detection using Faster R-CNN and increased mIoU by 0.7% for UpperNet on ADE20K segmentation.

Conclusions:

• Reverse Attention (RA) effectively enhances deep neural network performance by intelligently guiding information flow.
• The proposed RA-Net, incorporating M-GRN, offers a superior alternative to standard residual learning for computer vision applications.
• The method shows broad applicability and significant gains in classification, detection, and segmentation tasks.