Efficient Layer Compression Without Pruning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep Neural Networks (DNNs) require significant computational resources.
• Network pruning methods improve efficiency but often degrade performance by disrupting learned correlations.
• Existing compression techniques struggle to maintain network integrity during efficiency improvements.

Purpose of the Study:

• To propose a novel Efficient Layer Compression (ELC) method for deep neural networks.
• To reduce network depth and computational cost without performance loss.
• To enhance inference speed, particularly for edge devices.

Main Methods:

• Developed a novel decoupling module to separate network layers.
• Enabled merging of serial convolutional and nonlinear layers through decoupling.
• Achieved lossless compression via equivalent parameter conversion after merging.
• Applied the ELC approach to compress VGG-16 and ResNet-56 architectures.

Main Results:

• Achieved a 74.1% FLOPs reduction for VGG-16 and 54.6% for ResNet-56.
• Demonstrated superior performance retention compared to pruning methods.
• Improved inference speed by 2x on the Jetson AGX Xavier edge device.
• Successfully reduced network depth without destroying convolutional layer correlations.

Conclusions:

• Efficient Layer Compression (ELC) offers an effective strategy for lossless network compression.
• The decoupling and merging approach preserves network correlations, avoiding performance degradation.
• ELC significantly enhances computational efficiency and inference speed for DNNs, making them suitable for edge computing.