Analyzing Parameter-Efficient Convolutional Neural Network Architectures for Visual Classification

Advances in visual recognition have relied on increasingly deep and wide convolutional neural networks (CNNs), which often introduce substantial computational and memory costs. This review summarizes recent progress in parameter-efficient CNN design across three directions: hypercomplex representations with cross-channel weight sharing, axial attention mechanisms, and real-valued architectures using separable convolutions. We highlight how these approaches reduce parameter counts while maintaining or improving accuracy. We further analyze our contributions within this landscape. Full hypercomplex neural networks (FHNNs) employ hypercomplex layers throughout the architecture to reduce latency and parameters, while representational axial attention models (RepAA) extend this efficiency by generating additional feature representations. To mitigate the remaining overhead of spatial hypercomplex operations, we introduce separable hypercomplex networks (SHNNs), which factorize quaternion convolutions into sequential vectormap operations, lowering parameters by approximately 50%. Finally, we compare these models with popular efficient architectures, such as MobileNets and SqueezeNets, and demonstrate that our residual one-dimensional convolutional networks (RCNs) achieve competitive performance in image classification and super-resolution with significantly fewer parameters. This review highlights emerging strategies for reducing computational overhead in CNNs and outlines directions for future research.