Multiscale Hybrid Convolutional Deep Neural Networks with Channel Attention

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Attention mechanisms enhance neural network performance but increase computational overhead.
• Reducing model complexity while maintaining performance is a key research challenge.

Purpose of the Study:

• To propose a lightweight Mixture Attention (MA) module for improving neural network efficiency and effectiveness.
• To address the trade-off between performance gains and computational cost in attention networks.

Main Methods:

• A multi-branch architecture processes input feature maps for multi-scale information extraction.
• Group convolution is employed within each branch to minimize parameters.
• Channel attention is applied to fused feature maps for statistical information extraction.

Main Results:

• The MA module reduced network parameters by 9.86% and computational cost by 7.83%.
• Top-1 accuracy improved by 1.99% compared to ResNet50.
• Significant outperformance over state-of-the-art methods on CIFAR-10 and PASCAL-VOC benchmarks was demonstrated.

Conclusions:

• The proposed Mixture Attention module offers an efficient and effective solution for enhancing neural networks.
• MA achieves superior accuracy with reduced model complexity, making it suitable for resource-constrained applications.