SlimConv: Reducing Channel Redundancy in Convolutional Neural Networks by Features Recombining

Area of Science:

• Computer Vision
• Deep Learning Architectures

Background:

• Convolutional Neural Networks (CNNs) suffer from channel redundancy, leading to high memory and computational costs.
• Efficient CNN architectures are crucial for deploying deep learning models on resource-constrained devices.

Purpose of the Study:

• To design a novel module, Slim Convolution (SlimConv), to reduce channel redundancies in CNNs.
• To improve the efficiency and performance of CNN models through architectural optimization.

Main Methods:

• Developed a Slim Convolution (SlimConv) module with Reconstruct, Transform, and Fuse steps.
• Integrated SlimConv as a plug-and-play unit to replace standard convolutional layers.
• Validated SlimConv on ImageNet, MS COCO2014, Pascal VOC2012 segmentation, and Pascal VOC2007 detection datasets.

Main Results:

• SlimConv-equipped models consistently achieved better performance with reduced memory and computation.
• ResNet-101 with SlimConv reached 77.84% top-1 accuracy on ImageNet, using 3 GFLOPs less and 38% fewer parameters.
• Demonstrated significant model compression and performance enhancement across various benchmarks.

Conclusions:

• SlimConv effectively reduces channel redundancy in CNNs, leading to more efficient models.
• The proposed module offers a practical solution for compressing deep learning models without sacrificing accuracy.
• SlimConv presents a promising approach for developing lightweight yet powerful CNNs.