Spatially Adaptive Feature Refinement for Efficient Inference.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Convolutional Neural Networks (CNNs) often exhibit spatial redundancy in learned features.
• This redundancy leads to inefficient computations, particularly with high-resolution image data.

Purpose of the Study:

• To introduce a novel Spatially Adaptive feature Refinement (SAR) method.
• To reduce computational overhead in CNNs by addressing spatial redundancy.

Main Methods:

• SAR employs a dual-branch approach: one for standard convolution at lower resolution, and another for selective refinement at original resolution.
• It adaptively fuses features from both branches to minimize redundant calculations.

Main Results:

• SAR consistently enhances network performance and efficiency across various tasks (classification, object detection, segmentation).
• On ImageNet, SAR refined less than 40% of regions for 97% of samples, achieving comparable accuracy to the original ResNet model.
• Demonstrates significant computational redundancy in the spatial dimension of CNNs.

Conclusions:

• The proposed SAR method effectively reduces spatial redundancy in CNNs.
• SAR offers a flexible plug-and-play solution to improve the efficiency of existing CNN architectures.
• The findings highlight the potential for substantial computational savings in deep learning models.