Efficient and Flexible Method for Reducing Moderate-Size Deep Neural Networks with Condensation.

Area of Science:

• Artificial Intelligence
• Computational Science
• Machine Learning

Background:

• Neural networks show promise in scientific applications, but their moderate size impacts inference speed.
• Reducing neural network size is crucial for rapid computations in scientific tasks.
• Existing theories suggest neural network nonlinearity leads to neuron condensation, enabling size reduction.

Purpose of the Study:

• To propose and validate a condensation reduction method for neural networks in practical scientific problems.
• To demonstrate the feasibility of reducing neural network scale while maintaining performance.
• To confirm theoretical findings on neuron condensation through empirical evidence.

Main Methods:

• Developed a condensation reduction method applicable to fully connected and convolutional neural networks.
• Applied the method to complex combustion acceleration and CIFAR10 image classification tasks.
• Evaluated the impact of network size reduction on prediction accuracy and validation accuracy.

Main Results:

• Successfully reduced neural network size in combustion acceleration tasks to 41.7% of the original scale, maintaining accuracy.
• Achieved an 11.5% network size reduction in CIFAR10 image classification, with satisfactory validation accuracy.
• Demonstrated positive results across different network architectures and tasks.

Conclusions:

• The proposed condensation reduction method is effective for practical scientific applications.
• The method significantly reduces computational pressure and enhances inference speed for trained neural networks.
• This work validates the theory of neuron condensation and its utility in optimizing neural network performance.