LdsConv: Learned Depthwise Separable Convolutions by Group Pruning

Area of Science:

• Computer Vision
• Deep Learning Architectures
• Machine Learning Optimization

Background:

• Standard convolutional filters incur significant computational costs due to feature overlap.
• Existing convolutional neural network (CNN) architectures often exhibit inefficiencies in feature extraction.

Purpose of the Study:

• To propose a novel and efficient convolutional operation, Learned Depthwise Separable Convolution (LdsConv).
• To reduce computational cost and improve learning capacity in CNNs.

Main Methods:

• Developed LdsConv, a generic convolutional unit integrating pruning techniques.
• Replaced standard convolutions with LdsConv in state-of-the-art CNNs (ResNet, DenseNet, SE-ResNet, MobileNet).

Main Results:

• LdsConv significantly improves accuracy while reducing computational cost across various CNNs.
• ResNet50 with LdsConv achieved a 40.9% reduction in FLOPs and increased ImageNet accuracy.

Conclusions:

• LdsConv offers a direct and effective replacement for standard convolutions.
• The proposed method enhances CNN efficiency and performance without architectural modifications.