Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning.

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) offer high performance in computer vision but incur significant computational and memory costs.
• These costs limit CNN deployment on resource-constrained platforms like mobile and embedded systems.
• CNN compression research is crucial for enabling efficient AI on edge devices.

Purpose of the Study:

• To propose a novel filter pruning scheme, structured sparsity regularization (SSR), for CNN compression.
• To simultaneously reduce computation and memory overhead in CNNs.
• To develop an efficient optimization method for the proposed scheme.

Main Methods:

• Structured Sparsity Regularization (SSR) incorporates two structured sparsity regularizers into filter pruning.
• The Alternative Updating with Lagrange Multipliers (AULM) scheme efficiently solves the optimization problem.
• AULM alternates between promoting structured sparsity and optimizing recognition loss, inspired by ADMM.

Main Results:

• SSR adaptively prunes filters by coordinating global output and local pruning operations.
• The AULM solver is highly efficient, outperforming recent methods.
• The method significantly reduces the number of filters and feature maps, leading to memory-light inference.
• SSR achieves superior performance across various CNN architectures (LeNet, AlexNet, VGGNet, ResNet, GoogLeNet) and datasets.
• The scheme demonstrates strong performance gains in transfer learning tasks like domain adaptation and object detection.

Conclusions:

• Structured sparsity regularization (SSR) effectively compresses CNNs, reducing computational and memory demands.
• The proposed AULM optimization method provides an efficient solution for filter pruning.
• SSR enhances CNN applicability in resource-limited environments and shows promise in transfer learning applications.