Jump-GRS: a multi-phase approach to structured pruning of neural networks for neural decoding

Area of Science:

• Neural Engineering
• Machine Learning
• Computational Neuroscience

Background:

• Neural decoding links brain activity to behavior, with Deep Neural Networks (DNNs) showing promise.
• High decoding accuracy and real-time speed are crucial for applications like brain-computer interfaces.
• Existing pruning methods like Greedy inter-layer order with Random Selection (GRS) are computationally intensive for large-scale DNNs.

Purpose of the Study:

• To develop an efficient algorithm for compressing large-scale DNNs in neural decoding.
• To improve the computational efficiency of the GRS pruning method.
• Introduce jump Greedy inter-layer order with Random Selection (JGRS) for faster and scalable DNN compression.

Main Methods:

• Developed JGRS by incorporating a 'jump mechanism' into the GRS algorithm.
• The jump mechanism bypasses intermediate model retraining when accuracy is less sensitive to pruning.
• Identified pruning phases where retraining can be infrequent to enhance speed and scalability.

Main Results:

• JGRS demonstrated significantly faster pruning speeds compared to GRS.
• Pruned models generated by JGRS exhibited similar compactness to those from GRS.
• JGRS achieved 9%-20% more compressed models with 2-8 times faster execution across multiple models.

Conclusions:

• JGRS offers a computationally efficient and scalable solution for compressing large-scale DNNs in neural decoding.
• The jump mechanism effectively accelerates the pruning process without compromising model performance.
• JGRS is a viable alternative for applications requiring rapid DNN compression for neural data analysis.