Learn Singularly Perturbed Solutions via Homotopy Dynamics.

Area of Science:

• Scientific machine learning
• Numerical analysis
• Computational mathematics

Background:

• Neural networks are increasingly used for solving partial differential equations (PDEs).
• Training neural networks for singularly perturbed problems is difficult due to parameter-induced loss function singularities.
• Existing methods struggle with the near-singularities present in these problems.

Purpose of the Study:

• To develop a novel method for effectively training neural networks on singularly perturbed problems.
• To address the challenges posed by parameters that create near-singularities in the loss function.
• To theoretically analyze and experimentally validate a new optimization strategy.

Main Methods:

• Introduction of a novel method based on homotopy dynamics.
• Manipulation of parameters within the partial differential equations.
• Theoretical analysis of parameter effects on training difficulty and convergence.
• Experimental validation of the homotopy dynamics approach.

Main Results:

• The proposed homotopy dynamics method effectively manipulates problematic parameters.
• Theoretical convergence of the homotopy dynamics method is established.
• Significant acceleration in convergence and improved accuracy for singularly perturbed problems were demonstrated.
• The method provides an efficient optimization strategy.

Conclusions:

• Homotopy dynamics offers a robust framework for solving singularly perturbed differential equations with neural networks.
• The approach overcomes key training challenges associated with these problems.
• This work extends the applicability of neural networks in scientific machine learning for complex PDEs.