SymRefine: A symbolic regression approach for refining and compressing neural networks

Traditional methods for compressing neural networks (NN), such as pruning and distillation, primarily focus on reducing the size of the NN, often at the expense of accuracy. To overcome this limitation, we propose a novel refining NN method based on symbolic regression called SymRefine. SymRefine constructs a Cartesian genetic programming linear method (LCGP) to identify the mathematical expression that represents the relationship between the high-dimensional input and output of a selected module within the NN. The effectiveness of SymRefine was tested using two types of NN benchmarks: multilayer perceptron NNs (MLP) and convolutional NNs (CNN). Experimental results show that, compared with NN pruning and NN distillation, SymRefine can effectively reduce the number of the NN parameters while maintaining or even enhancing inference accuracy. This research demonstrates the potential of SymRefine as a novel tool for creating more efficient and accurate neural networks.