CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks often exhibit significant parametric and computational redundancy.
• Existing model pruning methods frequently rely on heuristics and overlook the joint impact of channels, leading to suboptimal performance.

Purpose of the Study:

• To propose a novel channel pruning method, class-aware trace ratio optimization (CATRO), for reducing computational burden and accelerating deep neural network inference.
• To address the limitations of existing empirical pruning approaches by considering the joint impact of channels.

Main Methods:

• CATRO utilizes class information from limited samples to measure the joint impact of multiple channels via feature space discriminations.
• Channel pruning is formulated as a submodular set function maximization problem, solved efficiently using a two-stage greedy iterative optimization.
• Theoretical justifications for the convergence of CATRO and the performance of pruned networks are provided.

Main Results:

• CATRO achieves higher accuracy with comparable computation costs or similar accuracy with reduced computation costs compared to state-of-the-art algorithms.
• The class-aware nature of CATRO enables adaptive pruning of efficient networks for diverse classification subtasks.

Conclusions:

• CATRO offers an effective and theoretically grounded approach to channel pruning for deep neural networks.
• The method enhances the practical deployment and usability of deep networks in real-world applications by enabling adaptive efficiency improvements.