Reduced storage direct tensor ring decomposition for convolutional neural networks compression.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning Optimization

Background:

• Convolutional Neural Networks (CNNs) are essential for computer vision tasks like image classification.
• Model compression is crucial for enhancing CNN efficiency in terms of storage and computation.
• Low-rank approximation methods offer a promising avenue for CNN compression by decomposing large kernels.

Purpose of the Study:

• To propose a novel low-rank method for CNN compression.
• To leverage reduced storage direct tensor ring decomposition (RSDTR) for efficient kernel approximation.
• To evaluate the effectiveness of RSDTR in achieving high compression rates and preserving accuracy.

Main Methods:

• Developed a novel low-rank CNN compression technique based on RSDTR.
• Implemented RSDTR to approximate convolutional kernels, reducing parameter and FLOPS complexity.
• Conducted experiments on CIFAR-10 and ImageNet datasets to assess performance.

Main Results:

• The proposed RSDTR method achieved significant parameter and FLOPS compression rates.
• RSDTR demonstrated superior circular mode permutation flexibility compared to existing methods.
• Compressed networks using RSDTR maintained competitive classification accuracy.

Conclusions:

• RSDTR is an effective method for compressing CNNs.
• The approach offers a favorable trade-off between compression efficiency and classification performance.
• RSDTR outperforms other state-of-the-art CNN compression techniques.