Training Compact CNNs for Image Classification Using Dynamic-Coded Filter Fusion.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Traditional filter pruning methods rely on pre-trained models or complex regularization.
• These methods often involve computationally intensive importance estimation or hyperparameter tuning.

Purpose of the Study:

• To introduce a novel, computation-economical, and regularization-free filter pruning method called dynamic-coded filter fusion (DCFF).
• To enable efficient image classification by deriving compact Convolutional Neural Networks (CNNs).

Main Methods:

• Filters are represented by inter-similarity distributions with a temperature parameter.
• A Kullback-Leibler divergence-based dynamic-coded criterion evaluates filter importance.
• Filter fusion, a weighted averaging of filters, is proposed for preserving important filters.

Main Results:

• DCFF successfully derives compact CNNs without dependency on pre-trained models or sparse constraints.
• Experiments show superior performance compared to existing methods on image classification benchmarks.
• A compact VGGNet-16 achieved 93.47% top-1 accuracy on CIFAR-10 with significantly reduced FLOPs and parameters.
• A compact ResNet-50 demonstrated substantial reductions in FLOPs and parameters while maintaining high accuracy on ILSVRC-2012.

Conclusions:

• DCFF offers an effective and efficient approach for developing compact CNNs.
• The dynamic nature of DCFF allows filter importance to adapt during training.
• This method provides a promising alternative for efficient deep learning model deployment.