Synthesis of recurrent neural networks for dynamical system simulation.

Area of Science:

• Computational Science
• Machine Learning
• Dynamical Systems Theory

Background:

• Recurrent neural networks (RNNs) are powerful tools for modeling complex systems.
• Approximating dynamical systems with neural networks is a challenging but crucial task.
• Existing training techniques for RNNs have limitations in approximating continuous-time dynamics.

Purpose of the Study:

• To review current methods for training RNNs to approximate dynamical systems.
• To introduce a novel algorithm for training RNNs that guarantees approximation quality.
• To demonstrate the capabilities of the new algorithm using numerical examples.

Main Methods:

• Review of established RNN training techniques for dynamical systems.
• Development of a novel algorithm based on theoretical guarantees.
• Training a feedforward neural network on a vector-field representation using backpropagation.
• Recasting the trained feedforward network into a recurrent network.
• Simulation of continuous-time dynamical systems using both the original systems and the trained RNNs.

Main Results:

• The proposed algorithm effectively trains recurrent neural networks to approximate dynamical systems.
• Numerical examples validate the algorithm's capabilities and the quality of network approximation.
• The approach allows both the dynamical systems and their neural network simulators to operate in continuous time.

Conclusions:

• The novel algorithm provides a robust method for approximating continuous-time dynamical systems with recurrent neural networks.
• This approach offers improved accuracy and a unified framework for continuous-time system simulation.
• The findings contribute to advancements in machine learning applications for scientific modeling.