An Attention-based Spatio-Temporal Neural Operator for Evolving Physics

Area of Science:

• Scientific Machine Learning (SciML)
• Physics-informed Machine Learning
• Computational Science

Background:

• Traditional machine learning models struggle with generalization in dynamic environments and lack physical interpretability.
• Learning evolving physical processes across spatio-temporal scales is crucial for applications like additive manufacturing.
• Adapting to unseen environmental parameters while capturing long-range interactions remains a key challenge.

Purpose of the Study:

• To propose a novel architecture, the Attention-based Spatio-Temporal Neural Operator (ASNO), for enhanced scientific machine learning.
• To improve the adaptability and interpretability of machine learning models in dynamic physical systems.
• To enable reliable predictions across spatio-temporal scales and generalize to unseen physical environments.

Main Methods:

• Developed the Attention-based Spatio-Temporal Neural Operator (ASNO) architecture.
• Integrated separable attention mechanisms for spatial and temporal interactions.
• Utilized a transformer for temporal prediction and an attention-based neural operator for external loads, inspired by the backward differentiation formula (BDF).

Main Results:

• ASNO demonstrated superior performance on scientific machine learning benchmarks compared to existing models.
• The architecture effectively captures long-range spatio-temporal interactions and adapts to unseen physical parameters.
• ASNO enhances interpretability by isolating historical state contributions and external forces.

Conclusions:

• ASNO offers a promising approach for learning unknown, evolving physical processes.
• The model shows significant potential for engineering applications, physics discovery, and interpretable machine learning.
• ASNO advances the capability of machine learning models to generalize and adapt in complex physical environments.