Gradient Statistics-Based Multi-Objective Optimization in Physics-Informed Neural Networks.

Area of Science:

• Computational Physics
• Machine Learning
• Numerical Analysis

Background:

• Modeling complex non-linear systems is crucial in science and engineering.
• Neural networks excel at learning from data but often require large datasets.
• Physics-Informed Neural Networks (PINNs) integrate domain knowledge (mathematical models) with neural networks to overcome data limitations.

Purpose of the Study:

• To address the challenge of optimizing multi-objective loss functions in PINNs.
• To propose and evaluate advanced gradient statistics-based weighting schemes for improved PINN training.
• To enhance the accuracy and reliability of PINNs in solving differential equations.

Main Methods:

• Developed novel weighting schemes for PINNs based on gradient statistics (kurtosis-standard deviation, mean-standard deviation).
• Utilized backpropagated gradient statistics to dynamically scale and weight individual loss terms.
• Applied proposed methods to solve 2D Poisson's and Klein-Gordon's equations.

Main Results:

• The proposed gradient statistics-based weighting schemes effectively balance the loss terms in PINNs.
• These advanced schemes lead to more stable training and improved accuracy compared to standard methods.
• Demonstrated significant improvements in approximating solutions for the tested differential equations.

Conclusions:

• Advanced gradient statistics-based weighting schemes are effective in overcoming optimization challenges in PINNs.
• The introduced kurtosis-standard deviation and combined mean-standard deviation schemes offer robust solutions for PDE approximation.
• These findings enhance the applicability of PINNs for complex scientific and engineering problems.