Learning stochastic dynamics and predicting emergent behavior using transformers.

Area of Science:

• Physics
• Artificial Intelligence
• Complex Systems

Background:

• Stochastic systems and active matter exhibit complex dynamics.
• Predicting emergent behavior in these systems often requires detailed knowledge of underlying rules.
• Neural networks, particularly transformers, show promise in learning complex patterns.

Purpose of the Study:

• To investigate if a transformer neural network can learn the dynamical rules of a stochastic system from limited data.
• To assess the predictive power of the trained network for emergent behavior under novel conditions.
• To explore the application of transformers in understanding complex physical systems.

Main Methods:

• Training a transformer neural network on a single trajectory of a lattice model of active matter.
• Simulating the active matter model at a specific density where it forms dispersed clusters.
• Evaluating the transformer's predictions on system behavior at densities not encountered during training.

Main Results:

• The transformer successfully learned the underlying dynamical rules of the stochastic system.
• The trained network accurately predicted motility-induced phase separation at new densities.
• The model demonstrated the capacity to represent numerous and nonlocal dynamical rules.

Conclusions:

• Transformers can learn complex system dynamics from observational data, even with a single trajectory.
• This approach enables accurate prediction of emergent behavior and phase transitions in unseen conditions.
• The method offers a flexible way to study diverse physical systems without explicit rate enumeration or coarse-graining.