Automatic network structure discovery of physics informed neural networks via knowledge distillation

Area of Science:

• Computational physics
• Machine learning for scientific modeling

Background:

• Partial differential equations (PDEs) are crucial for modeling physical phenomena.
• Current physics-informed neural networks (PINNs) struggle to automatically discover and embed physical structures due to reliance on external loss functions.

Purpose of the Study:

• To develop a method for automatic discovery and embedding of physically consistent structures in neural networks for PDEs.
• To improve the accuracy, efficiency, and adaptability of neural network models for scientific applications.

Main Methods:

• Physics-informed distillation to decouple physical and parameter regularization.
• Staged optimization using teacher and student networks.
• Clustering and parameter reconstruction for structure extraction.

Main Results:

• Successfully extracted relevant physical structures from PDEs.
• Demonstrated improved accuracy and training efficiency compared to traditional methods.
• Showcased enhanced structural adaptability and transferability across different physics problems.

Conclusions:

• The proposed method offers a new perspective for efficient modeling and automatic discovery of structured neural networks for PDEs.
• This approach facilitates the creation of more interpretable and robust physics-informed machine learning models.