Mask-PINNs: Mitigating internal covariate shift in physics-informed neural networks

Area of Science:

• Computational physics
• Machine learning for scientific computing

Background:

• Physics-Informed Neural Networks (PINNs) solve differential equations by integrating physical laws into the loss function.
• Internal Covariate Shift (ICS) is a key optimization challenge in PINNs, degrading training stability and model performance.
• Standard ICS normalization techniques like Batch Normalization are incompatible with PINNs' deterministic requirements.

Purpose of the Study:

• To introduce a novel method, Mask-PINNs, to mitigate Internal Covariate Shift (ICS) in Physics-Informed Neural Networks (PINNs).
• To develop a technique that stabilizes PINN training without disrupting the core physics-based formulation.
• To enhance the expressiveness and applicability of PINNs.

Main Methods:

• Proposed Mask-PINNs, incorporating a smooth, learnable mask to adaptively regulate internal network features.
• Developed a theoretical framework demonstrating the mask's ability to suppress feature representation expansion via modulation.
• Validated Mask-PINNs across various partial differential equation (PDE) benchmarks and activation functions.

Main Results:

• Mask-PINNs demonstrated consistent improvements in prediction accuracy, convergence stability, and robustness.
• The method effectively addressed Internal Covariate Shift (ICS) without compromising the physics-informed loss.
• Enabled the use of wider network architectures, overcoming a limitation of conventional PINN frameworks.

Conclusions:

• Mask-PINNs offer a robust solution to Internal Covariate Shift (ICS) in Physics-Informed Neural Networks (PINNs).
• The proposed masking strategy enhances training dynamics and model performance for solving differential equations.
• Mask-PINNs represent a significant advancement for applying neural networks in scientific computing.