Coupled Tensor Decomposition for Compact Network Representation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional layers in deep learning models contain redundant filters that produce overlapping outputs.
• This redundancy leads to increased model parameters and computational complexity.

Purpose of the Study:

• To introduce a novel filter decomposition method that exploits redundancy in convolutional filters.
• To reduce model size and computational cost while maintaining performance.

Main Methods:

• Proposing coupled filters decomposition using coupled canonical polyadic decomposition (CPD).
• Implementing filter clustering based on a custom metric before decomposition to enhance efficiency.
• Applying a less restrictive coupling constraint within identified filter groups.

Main Results:

• The coupled filters decomposition method significantly reduces model parameters and computational complexity.
• Experimental validation across diverse architectures, datasets, and tasks shows competitive performance against state-of-the-art compression techniques.
• The approach effectively addresses filter redundancy in convolutional neural networks.

Conclusions:

• Coupled filters decomposition is an effective technique for compressing deep learning models.
• The method offers a promising direction for efficient neural network design and deployment.
• The proposed approach achieves a favorable trade-off between model compression and performance.