Bayesian polynomial neural networks and polynomial neural ordinary differential equations

Area of Science:

• Computational Science
• Machine Learning
• Applied Mathematics

Background:

• Symbolic regression using polynomial neural networks and polynomial neural ordinary differential equations (ODEs) are advanced techniques for scientific equation discovery.
• Current methods offer point estimates, limiting their applicability with real-world, noisy datasets.

Purpose of the Study:

• To develop and validate Bayesian inference methods for symbolic regression that can effectively handle noisy data.
• To compare the performance of Laplace approximation, Markov Chain Monte Carlo (MCMC) sampling, and variational inference for this task.

Main Methods:

• Implementation of Bayesian inference techniques: Laplace approximation, MCMC sampling, and variational inference.
• Application of these methods to polynomial neural networks and polynomial ODEs for equation recovery.
• Validation of the developed methods on problems with noisy data.

Main Results:

• The Laplace approximation demonstrated superior performance compared to MCMC and variational inference for the tested class of problems.
• Successful accommodation of noisy data in symbolic regression tasks.
• The developed Bayesian framework is adaptable to broader symbolic neural network architectures.

Conclusions:

• Bayesian inference, particularly the Laplace approximation, significantly enhances the robustness of polynomial neural networks and ODEs for symbolic regression with noisy data.
• This work provides a more reliable approach for equation discovery in science and engineering.
• The methodology is extensible to a wider range of symbolic neural network models.