Training stiff neural ordinary differential equations with explicit exponential integration methods

Area of Science:

• Computational science
• Machine learning
• Applied mathematics

Background:

• Stiff ordinary differential equations (ODEs) are prevalent in science and engineering.
• Standard neural ODEs struggle with stiff systems, limiting their application.
• Previous work used computationally expensive implicit methods for stiff neural ODEs.

Purpose of the Study:

• To explore explicit exponential integration methods as a more efficient alternative for stiff neural ODEs.
• To evaluate the performance of explicit methods in handling stiff dynamics.
• To improve the applicability of neural ODEs to scientific and engineering problems.

Main Methods:

• Investigated explicit exponential integration methods.
• Evaluated the integrating factor Euler (IF Euler) method.
• Compared performance against implicit methods on the stiff van der Pol oscillator.

Main Results:

• The IF Euler method demonstrated superior stability and efficiency compared to implicit methods.
• IF Euler successfully trained the stiff van der Pol oscillator, unlike implicit schemes.
• Large step sizes were feasible with the IF Euler method.

Conclusions:

• Explicit exponential integration, particularly IF Euler, is a viable and efficient approach for stiff neural ODEs.
• First-order accuracy of IF Euler presents a limitation.
• Developing higher-order explicit methods for stiff neural ODEs remains an open research challenge.