Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks

Area of Science:

• Computational Physics
• Scientific Machine Learning
• Numerical Analysis

Background:

• Inverse problems in physics are often modeled by partial differential equations (PDEs).
• Neural networks (NNs) have been applied to solve these inverse problems by minimizing PDE-based loss functions.
• Existing NN approaches face limitations in computational speed and accuracy.

Purpose of the Study:

• To introduce a novel framework, Optimizing a Discrete Loss (ODIL), for significantly accelerating the solution of inverse problems governed by PDEs.
• To demonstrate that ODIL outperforms physics-informed neural networks (PINNs) in terms of computational speed, accuracy, and convergence rates.
• To provide a powerful tool that bridges numerical methods and machine learning for scientific applications.

Main Methods:

• Developed the Optimizing a Discrete Loss (ODIL) framework, which uses discrete approximations of PDEs instead of NNs.
• Employed gradient-based and Newton's methods for minimizing a discrete cost function.
• Integrated machine learning tools for automatic differentiation and a multigrid technique to accelerate convergence.

Main Results:

• Achieved a five-orders-of-magnitude acceleration in solving inverse problems compared to NN-based methods.
• ODIL demonstrated superior accuracy and convergence rates over physics-informed neural networks.
• Successfully applied ODIL to various problems, including PDE-constrained optimization, optical flow, system identification, data assimilation, and Navier-Stokes equations.

Conclusions:

• ODIL offers a computationally efficient and accurate alternative to NN-based methods for solving inverse problems in physics.
• The framework inherits the desirable properties of grid-based PDE discretizations, such as accuracy and conservation.
• ODIL represents a significant advancement, bridging numerical methods and machine learning for broad scientific discovery.