Neural Network with Optimal Neuron Activation Functions Based on Additive Gaussian Process Regression.

Area of Science:

• Computational chemistry
• Materials informatics
• Physical chemistry

Background:

• Neural networks (NNs) are widely used in science but limited by simple, uniform neuron activation functions.
• Enhanced flexibility in activation functions can reduce computational cost and improve NN expressive power.

Purpose of the Study:

• To develop a method for constructing optimal, individual neuron activation functions using additive Gaussian process regression (GPR).
• To integrate this approach into a framework that avoids nonlinear fitting, combining linear regression robustness with NN expressive power.

Main Methods:

• Utilized additive Gaussian process regression (GPR) to generate unique activation functions for each neuron.
• Developed a rule-based system to define neural network parameters, bypassing nonlinear fitting.
• Applied the method to fit potential energy surfaces for water and formaldehyde molecules.

Main Results:

• The additive-GPR-based method demonstrated superior performance compared to conventional NNs in high-accuracy fitting tasks.
• The approach effectively mitigated overfitting issues common in conventional NNs.
• Achieved high accuracy without the need for computationally expensive nonlinear optimization.

Conclusions:

• Additive GPR offers a robust and efficient way to enhance neural network capabilities for scientific modeling.
• This novel method provides a powerful alternative to conventional neural networks, particularly in data-intensive scientific domains.
• The technique successfully models complex potential energy surfaces with improved accuracy and reduced computational burden.