Data-driven learning of chaotic dynamical systems using Discrete-Temporal Sobolev Networks

Area of Science:

• Computational Science
• Applied Mathematics
• Machine Learning

Background:

• Forecasting dynamical systems is challenging, particularly for chaotic systems sensitive to initial conditions.
• Existing neural network approaches often struggle with noise and require derivative information.

Purpose of the Study:

• Introduce the Discrete-Temporal Sobolev Network (DTSN) as a novel loss function for dynamical system forecasting.
• Evaluate DTSN's effectiveness compared to standard Mean Squared Error (MSE) and Physics-Informed Neural Network (PINN) losses.

Main Methods:

• Developed DTSN, a data-driven and architecture-agnostic loss function minimizing variational differences using a temporal Sobolev norm.
• Applied DTSN with Long Short-Term Memory (LSTM) and Transformer architectures to discrete approximations of the Lorenz-63 and Chua circuit systems.

Main Results:

• DTSN significantly improved forecasting accuracy for both LSTM and Transformer architectures.
• DTSN demonstrated superior performance over MSE loss and required less information than PINN loss.
• Computational time was not noticeably increased by using DTSN.

Conclusions:

• DTSN offers a powerful, data-driven method for enhancing neural network-based forecasting of dynamical systems.
• The approach is particularly beneficial for chaotic systems due to its noise-minimizing properties.
• DTSN presents a viable alternative to existing methods, improving accuracy without significant computational overhead.