RASP: Regularization-based Amplitude Saliency Pruning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Existing filter pruning methods often rely on data-dependent criteria, limiting their theoretical feasibility and practical deployment on resource-constrained devices.
• Norm criteria based on amplitude measurements face challenges in theoretical grounding and quantitative analysis of filter importance.
• Data-derived information used in current pruning standards hinders true data independence and robust evaluation.

Discussion:

• The proposed regularization-based amplitude saliency pruning (RASP) criterion offers data independence and theoretical feasibility for filter pruning.
• RASP establishes clear norm criteria for filter importance, addressing the 'smaller-norm-less-important' challenge.
• Model saliency and adaptive parameter group lasso (AGL) regularization are introduced to tackle data dependency in evaluation and inter-class filter selection.

Key Insights:

• RASP achieves data independence and theoretical feasibility in filter pruning, outperforming previous methods.
• Quantitative saliency analysis validates the advantages of RASP over existing approaches.
• RASP enables significant reduction in computational complexity (FLOPs) while maintaining or improving model accuracy.

Outlook:

• RASP demonstrates strong potential for deploying efficient deep neural networks on edge devices and other resource-limited platforms.
• Further research can explore adaptive parameter group lasso (AGL) for layer-specific optimization in pruning.
• The validated amplitude saliency offers a promising direction for developing more robust and theoretically sound pruning techniques.