Redundant feature pruning for accelerated inference in deep neural networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep and wide convolutional neural network models often contain redundant features, leading to increased computational costs.
• Over-sized models exhibit feature redundancy, where filters are similar or shifted versions of each other, impacting efficiency.

Purpose of the Study:

• To present an efficient technique for reducing the inference cost of deep and wide convolutional neural network models.
• To address the issue of filtering redundancy in oversized deep neural network models.

Main Methods:

• Proposing a method to prune redundant features (filters) and their associated feature maps.
• Utilizing relative cosine distances in the feature space to identify and remove redundant filters.

Main Results:

• Demonstrated significant reduction in inference costs (in FLOPS) across various models (VGG-16, ResNet-56, ResNet-110, ResNet-34).
• Achieved competitive performance comparable to state-of-the-art models on datasets like MNIST, CIFAR-10, and ImageNet.
• Resulted in smaller network sizes with reduced post-training inference computational costs.

Conclusions:

• The proposed pruning technique effectively reduces computational inference costs in deep neural networks.
• The method maintains competitive performance, making it a viable approach for efficient model deployment.
• Feature pruning based on cosine distance is an effective strategy for optimizing convolutional neural network efficiency.