Packing Convolutional Neural Networks in the Frequency Domain.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) offer advanced capabilities but face challenges with storage and computation on mobile devices.
• Existing CNN compression methods often focus solely on weight reduction, limiting overall efficiency.

Purpose of the Study:

• To develop novel frequency domain techniques for compressing and accelerating CNNs.
• To enable the deployment of complex CNN models on resource-constrained mobile platforms.

Main Methods:

• Convolution filters are treated as images and decomposed into shared and individual frequency domain components.
• Low-energy frequency coefficients are discarded, and redundancies are removed in both spatial and frequency domains.
• A data-driven approach optimizes sparse CNNs in the frequency domain, utilizing Discrete Cosine Transform (DCT) bases for computational efficiency.

Main Results:

• Achieved high compression ratios with minimal impact on accuracy by discarding low-energy frequency coefficients.
• Demonstrated significant speed-up in convolution operations through linear combination of DCT bases.
• Evaluated performance on benchmark image datasets, showing superior compression and speed-up compared to state-of-the-art methods.

Conclusions:

• The proposed frequency domain approach effectively compresses and accelerates CNNs for mobile applications.
• This method addresses the computational and storage limitations of CNNs, broadening their applicability.
• The technique offers a promising solution for efficient deep learning on edge devices.