Analyzing Parameter-Efficient Convolutional Neural Network Architectures for Visual Classification.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning Architectures

Background:

• Deep and wide convolutional neural networks (CNNs) dominate visual recognition but incur high computational and memory costs.
• Parameter efficiency is crucial for deploying CNNs on resource-constrained devices.
• Existing efficient architectures like MobileNets and SqueezeNets offer trade-offs between efficiency and accuracy.

Purpose of the Study:

• To review recent advancements in parameter-efficient CNN design.
• To introduce novel hypercomplex and attention-based CNN architectures.
• To analyze the performance of these efficient models against established benchmarks.

Main Methods:

• Exploration of hypercomplex representations with cross-channel weight sharing.
• Implementation of axial attention mechanisms for feature representation.
• Development of real-valued architectures utilizing separable convolutions.
• Introduction of Full Hypercomplex Neural Networks (FHNNs), Representational Axial Attention (RepAA) models, and Separable Hypercomplex Networks (SHNNs).
• Factorization of quaternion convolutions into sequential vectormap operations in SHNNs.
• Comparison with MobileNets and SqueezeNets using Residual One-Dimensional Convolutional Networks (RCNs).

Main Results:

• Hypercomplex representations, axial attention, and separable convolutions reduce parameter counts effectively.
• FHNNs and RepAA models demonstrate enhanced efficiency.
• SHNNs achieve approximately 50% parameter reduction by factorizing quaternion convolutions.
• RCNs show competitive performance in image classification and super-resolution with significantly fewer parameters compared to existing models.

Conclusions:

• Parameter-efficient CNN designs are vital for advancing visual recognition.
• Novel hypercomplex and attention-based architectures offer significant reductions in computational overhead.
• Future research should focus on further optimizing CNNs for efficiency without compromising accuracy.