FPWT: Filter pruning via wavelet transform for CNNs.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) demand significant computational resources, limiting their use on mobile devices.
• Current filter pruning methods primarily focus on the spatial domain, overlooking interconnections and energy distribution.
• Image frequency domain transforms offer compression by concentrating energy, a principle explored for CNN feature maps.

Purpose of the Study:

• To introduce a novel filter pruning method using wavelet transform (WT) for CNNs.
• To leverage frequency domain information for more effective feature map analysis and pruning.
• To reduce the computational and parameter load of CNNs for practical mobile applications.

Main Methods:

• Developed Filter Pruning via Wavelet Transform (FPWT) combining WT with CNN feature maps.
• Calculated feature map importance using cosine similarity and energy-weighted frequency components.
• Pruned filters based on calculated importance scores.

Main Results:

• FPWT significantly reduces FLOPs and parameters in CNNs.
• On ResNet-110 (CIFAR-10), achieved >60% reduction in FLOPs/parameters with a 0.53% accuracy increase.
• On ResNet-50 (ImageNet), achieved 53.8% FLOPs reduction and 49.7% parameter reduction with <1% Top-1 accuracy loss.

Conclusions:

• FPWT effectively prunes CNN filters by analyzing frequency domain characteristics.
• The method offers substantial computational savings with minimal impact on accuracy.
• FPWT demonstrates practical applicability for deploying efficient CNNs on resource-constrained devices.