A Dual-Dimer method for training physics-constrained neural networks with minimax architecture

Area of Science:

• Engineering
• Scientific Computing
• Machine Learning

Background:

• Data sparsity is a significant challenge in training machine learning models for expensive scientific and engineering applications.
• Physics-Constrained Neural Networks (PCNNs) offer a solution by reducing data requirements, but their loss weights are often tuned empirically.
• Existing PCNN methods lack systematic approaches for optimizing the balance between data-driven and physics-based losses.

Purpose of the Study:

• To introduce a novel physics-constrained neural network with a minimax architecture (PCNN-MM) for systematic adjustment of loss function weights.
• To develop an efficient saddle point search algorithm for training PCNN-MMs.
• To demonstrate improved convergence and computational efficiency compared to traditional PCNNs.

Main Methods:

• A minimax architecture (PCNN-MM) was developed to systematically adjust weights for data and physical constraint losses.
• A novel saddle point search algorithm, the Dual-Dimer method, was created for training the PCNN-MM.
• The Dual-Dimer method was compared against gradient descent ascent for nonconvex-nonconcave functions.

Main Results:

• The PCNN-MM allows for systematic, rather than empirical, adjustment of loss weights.
• The Dual-Dimer method demonstrates superior computational efficiency compared to gradient descent ascent for high-order saddle point searches.
• The Dual-Dimer method provides eigenvalue information for verifying search results.
• A heat transfer example showed faster convergence for PCNN-MMs compared to traditional PCNNs.

Conclusions:

• The proposed PCNN-MM with the Dual-Dimer method offers a systematic and efficient approach to training physics-constrained neural networks.
• This method addresses the empirical weight tuning limitations of traditional PCNNs.
• The findings suggest potential for broader application in data-sparse scientific and engineering domains.