Towards Convolutional Neural Network Acceleration and Compression Based on Simonk-Means.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) are essential for image classification and target recognition but require significant computational resources.
• Model compression is crucial for deploying CNNs on embedded devices, yet traditional retraining methods are time-consuming and can reduce precision.
• Existing compression techniques often struggle to balance efficiency with performance, especially for hardware acceleration.

Purpose of the Study:

• To propose a novel model compression approach for CNNs that supports hardware acceleration.
• To introduce an extended k-means algorithm, Simon k-means, for clustering neural network weights.
• To reduce hardware resource consumption in data movement and storage for compressed CNNs.

Main Methods:

• Developed an extension algorithm named Simon k-means, based on the simple k-means clustering algorithm.
• Applied Simon k-means to cluster trained weights in convolutional and fully connected layers of CNNs.
• Implemented a data storage and index approach to minimize hardware resource consumption.
• Designed and provided a hardware implementation of the compressed CNN accelerator.

Main Results:

• Achieved a compression ratio of 5.27× in AlexNet on the FASHION-MNIST dataset.
• Reduced multiply-accumulate (MAC) operations by 74.3%.
• Demonstrated the effectiveness of the Simon k-means approach for CNN model compression and hardware acceleration.

Conclusions:

• The proposed Simon k-means based model compression technique offers significant efficiency gains for CNNs.
• This method effectively reduces computational complexity and hardware resource requirements, making CNNs more suitable for embedded systems.
• The hardware implementation validates the practical applicability of the compression scheme for real-world applications.