Explaining the physics of transfer learning in data-driven turbulence modeling

Area of Science:

• Scientific Machine Learning (ML)
• Dynamical Systems Theory
• Computational Fluid Dynamics

Background:

• Transfer learning (TL) is crucial for neural networks (NNs) to generalize in scientific ML, but optimal re-training strategies and learned physics remain unclear.
• Applications span weather/climate prediction and turbulence modeling, highlighting the need for effective TL methods.

Purpose of the Study:

• To develop a framework for understanding and optimizing TL in multi-scale, nonlinear dynamical systems.
• To identify the best re-training procedures by linking NN spectral properties to physical system dynamics.
• To explain the physics learned by NNs during TL.

Main Methods:

• Combined spectral analyses of dynamical systems (e.g., Fourier analysis) with spectral analyses of convolutional NNs.
• Developed a physics-guided framework integrating system and NN spectral properties.
• Applied the framework to subgrid-scale modeling in 2D turbulence simulations.

Main Results:

• Revealed physical connections between dynamical systems and NN learned features (filters).
• Demonstrated that shallowest convolution layers are optimal for re-training in 2D turbulence, challenging common ML practices.
• Validated the physics-guided framework for identifying optimal TL procedures.

Conclusions:

• The proposed framework enables optimal and explainable TL for scientific ML.
• This work advances the development of explainable NNs for science and engineering, including climate change modeling.
• Identified specific NN layers for re-training based on physical principles, offering a new direction for TL research.