Bridging known and unknown dynamics by transformer-based machine-learning inference from sparse observations

Area of Science:

• Nonlinear dynamics
• Machine learning
• Complex systems

Background:

• Accurate system dynamics reconstruction is crucial for many applications.
• Challenges arise when dealing with novel systems and sparse, one-time observations.
• Existing methods struggle with data scarcity and lack of prior system knowledge.

Purpose of the Study:

• To develop a novel machine learning framework for reconstructing complex nonlinear dynamics.
• To address the challenge of system identification with limited and sparse observational data.
• To enable faithful dynamics reconstruction when training data from the target system is unavailable.

Main Methods:

• A hybrid approach combining transformer networks and reservoir computing was developed.
• Transformers were trained on synthetic data from known chaotic systems.
• The trained transformer processed sparse data from the target system, feeding into a reservoir computer for prediction.

Main Results:

• The hybrid framework successfully reconstructed dynamics from reasonably sparse data across various nonlinear systems.
• Demonstrated the capability to predict long-term dynamics and attractors.
• Validated the model's effectiveness on prototypical nonlinear systems.

Conclusions:

• The proposed hybrid machine learning framework offers a novel paradigm for reconstructing complex nonlinear dynamics.
• It effectively handles situations with non-existent training data and sparse, random observations.
• This approach enables faithful dynamics reconstruction in previously unencountered systems.