DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Vision Transformers.

Area of Science:

• Deep Learning
• Computer Vision
• Model Compression

Background:

• Dynamic networks offer theoretical speedups but suffer from practical inefficiencies due to sparsity.
• Existing model compression methods often struggle to balance efficiency and accuracy.

Purpose of the Study:

• To introduce dynamic weight slicing for hardware-efficient dynamic inference in Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs).
• To improve the practical runtime acceleration of dynamic neural networks.

Main Methods:

• Dynamic weight slicing pre-defines dense weight slices with varying importance levels using nested residual learning.
• Introduced DS-CNN++ and DS-ViT++ architectures with novel dynamic gates and dynamic idle slicing.
• Proposed a two-stage optimization scheme including in-place bootstrapping (IB), multi-view consistency (MvCo), and sandwich gate sparsification (SGS).

Main Results:

• Demonstrated consistent outperformance over state-of-the-art static and dynamic model compression methods (up to 6.6% improvement).
• Achieved 2-4x computation reduction and up to 61.5% real-world acceleration on MobileNet, ResNet-50, and Vision Transformer.
• Minimal accuracy degradation observed on ImageNet.

Conclusions:

• Dynamic weight slicing presents a viable and effective approach for efficient deep learning inference.
• The proposed DS-CNN++ and DS-ViT++ models offer significant practical acceleration with high accuracy.
• The developed optimization strategies ensure sub-network generality and routing fairness for robust performance.