Loss-driven dynamic weight and residual transformation in physics-informed neural network

Area of Science:

• Computational physics
• Applied mathematics
• Machine learning

Background:

• Physics-Informed Neural Networks (PINNs) are increasingly used for forward and inverse problems governed by partial differential equations (PDEs).
• A key challenge in PINN training is the imbalance within the empirical risk loss function, hindering optimal performance.
• Understanding PINN failure modes, especially in high-dimensional inverse problems, is crucial for reliable scientific discovery.

Purpose of the Study:

• To identify and address the loss function imbalance in PINNs.
• To develop novel PINN methodologies that enhance accuracy and computational efficiency.
• To provide theoretical insights into PINN optimization and failure mechanisms for complex PDEs.

Main Methods:

• Introduction of a loss-driven dynamic weight PINN with theoretical analysis using the neural tangent kernel.
• Comprehensive evaluations on complex, time-dependent physical phenomena with varied learning rate decay strategies.
• Development of the Residual Transformation PINN (ResTranPINN) algorithm based on limiting equivalence for accelerated convergence.

Main Results:

• Demonstrated effectiveness of the dynamic weight PINN across diverse physical problems.
• Quantified the impact of dynamic weight update frequencies and allocation on accuracy and efficiency.
• The ResTranPINN algorithm significantly accelerated convergence and reduced computational time for inverse problems.

Conclusions:

• The proposed dynamic weight and ResTranPINN methods offer robust solutions to PINN training challenges.
• These advancements provide a deeper understanding of PINN optimization and failure modes.
• The study establishes a comprehensive framework for improving the reliability and efficiency of PINNs in scientific applications.