Training stiff neural ordinary differential equations with explicit rational Taylor series methods.

Area of Science:

• Numerical Analysis
• Machine Learning
• Dynamical Systems

Background:

• Stiff neural ordinary differential equations (NODEs) present significant computational challenges.
• Traditional implicit methods for stiff NODEs are often computationally intensive.
• Efficient and stable training of stiff dynamics is crucial for data-driven simulations.

Purpose of the Study:

• To introduce novel explicit rational Taylor series methods for directly training stiff NODEs.
• To enhance the efficiency and numerical stability of learning stiff dynamical systems.
• To provide a computationally reduced alternative to traditional implicit methods.

Main Methods:

• Development of second and third-order explicit rational Taylor series schemes.
• Demonstration of A-stability for the proposed explicit schemes.
• Application of these methods to train stiff systems, including the van der Pol oscillator.

Main Results:

• The explicit schemes achieve high efficiency with a single linear solve per time step.
• Proposed methods exhibit strong numerical stability, even at large step sizes.
• Effective learning of stiff dynamics was demonstrated without stability issues common to implicit schemes.
• Significantly reduced computational cost compared to traditional implicit methods.

Conclusions:

• Explicit rational Taylor methods provide an efficient and stable approach for training stiff NODEs.
• These methods expand the capabilities of data-driven simulation for complex dynamics.
• The findings support applications in mesh-based simulation and physics-informed neural networks.