Dense skip-attention for convolutional networks.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Attention mechanisms are vital for enhancing model performance by focusing on salient features.
• Current methods often overlook interactions between attention features across different modules.
• This limitation hinders the full potential of attention in complex network architectures.

Purpose of the Study:

• To propose a novel dense skip-attention method for convolutional networks.
• To enable learning of interactive attention features across all modules.
• To enhance the performance of existing attention mechanisms in computer vision tasks.

Main Methods:

• Developed a dense skip-attention approach connecting all attention modules.
• Integrated this method into convolutional network architectures.
• Conducted experiments on ImageNet 2012 and MS COCO 2017 datasets.

Main Results:

• The dense skip-attention method significantly improved performance in image classification, object detection, and instance segmentation.
• Effectiveness demonstrated in enhancing Squeeze-and-Excitation Networks, Efficient Channel Attention Networks, and Convolutional Block Attention Module.
• Achieved performance gains without substantial increases in model parameters or computational cost.

Conclusions:

• The proposed dense skip-attention method is effective in boosting convolutional network performance.
• It successfully captures interactive attention features, overcoming limitations of prior methods.
• This approach offers an efficient way to enhance attention mechanisms in deep learning models.