LAP: Latency-aware automated pruning with dynamic-based filter selection.

Area of Science:

• Deep Learning
• Computer Vision
• Model Compression

Background:

• Convolutional Neural Networks (CNNs) are computationally intensive.
• Model pruning techniques aim to reduce network size and improve inference speed.
• Existing methods often prioritize accuracy over latency optimization.

Purpose of the Study:

• To develop a novel automated pruning framework that optimizes both model accuracy and inference latency.
• To introduce 'latency sensitivity' as a key metric for guiding the pruning process.
• To enhance the efficiency of CNNs for real-world applications.

Main Methods:

• Developed the Latency-Aware Automated Pruning (LAP) framework utilizing reinforcement learning.
• Incorporated 'latency sensitivity' as prior knowledge within the reinforcement learning exploration loop.
• Designed a novel filter selection algorithm to identify critical filters based on dynamic changes.
• Trained the agent using feedback on accuracy error and latency sensitivity.

Main Results:

• LAP framework demonstrated superior performance over state-of-the-art compression policies on VGGNet, ResNet, and MobileNet.
• Achieved approximately 1.64x inference latency speedup for MobileNet-V1 on a Titan RTX GPU without compromising ImageNet Top-1 accuracy.
• Significantly improved the accuracy-latency trade-off Pareto curve.

Conclusions:

• The LAP framework effectively balances accuracy and latency optimization in CNN pruning.
• Latency sensitivity is a crucial factor for guiding automated pruning strategies.
• This approach offers a more comprehensive solution for efficient deep learning model deployment.