Spatial Re-Parameterization for N:M Sparsity.

Area of Science:

• Deep Learning
• Computer Vision
• Model Compression

Background:

• N:M sparsity offers structured compression but has limited spatial sparsity variation.
• Unstructured sparsity provides greater spatial sparsity variation, crucial for performance.
• Existing N:M methods struggle to match the performance of unstructured sparsity.

Purpose of the Study:

• To develop a method that enhances N:M sparsity by incorporating the spatial sparsity distribution of unstructured sparsity.
• To improve the performance of N:M sparse networks without introducing additional computational overhead during inference.

Main Methods:

• Introduced Spatial Re-parameterization (SpRe) method for N:M sparsity.
• Employed an auxiliary branch during training to mimic unstructured sparsity's spatial distribution.
• Developed an inference-time re-parameterization technique to merge the auxiliary branch without altering the N:M sparse pattern or computational cost.

Main Results:

• SpRe enables N:M sparse networks to achieve spatial sparsity distributions similar to unstructured sparsity.
• The method successfully matched the performance of state-of-the-art unstructured sparsity techniques.
• Performance was validated across various benchmarks, demonstrating broad applicability.

Conclusions:

• SpRe effectively bridges the performance gap between N:M and unstructured sparsity.
• The proposed method offers efficient model compression by leveraging the benefits of both sparsity types.
• This technique provides a promising direction for optimizing deep learning models.