Attention Inspiring Receptive-Fields Network for Learning Invariant Representations.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Spatial attention mechanisms are crucial for image classification.
• Existing methods may not fully leverage spatial attention for enhanced invariance.
• Convolutional Neural Networks (CNNs) benefit from improved feature representation.

Purpose of the Study:

• Introduce a novel plug-in module, Attention Inspiring Receptive-fields (Air), for image classification.
• Investigate the relationship between spatial attention and receptive fields.
• Enhance translation and scale invariance in deep learning models.

Main Methods:

• Developed the Air module by converting spatial attention into a plug-in component.
• Integrated the Air module into advanced CNN architectures (ResNet, ResNeXt) to create AirNet.
• Conducted extensive experiments on CIFAR and ImageNet datasets.

Main Results:

• AirNet architectures demonstrated significant improvements in image classification accuracy.
• AirNet-50 and AirNet-101 achieved 1.69% and 1.50% top-1 accuracy gains on ImageNet, respectively.
• The Air module enhances feature invariance with minimal increase in computation and parameters.

Conclusions:

• The Air module is an effective and efficient plug-in for improving CNN performance.
• AirNet facilitates learning invariant representations, boosting performance on challenging datasets.
• AirNet shows promise for transfer learning applications in object detection, segmentation, and pose estimation.