Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

Area of Science:

• Computational Science and Engineering (CSE)
• Numerical Analysis
• Deep Learning (DL)

Background:

• Deep Learning (DL) is increasingly applied in Computational Science and Engineering (CSE), showing promise for revolutionizing scientific computing.
• However, DL's reliability and efficiency, particularly regarding numerical analysis aspects like stability, robustness, accuracy, and sample complexity, remain poorly understood.
• Approximating solutions to parametric Partial Differential Equations (PDEs) is crucial for Uncertainty Quantification (UQ) in CSE, but often involves scarce, error-prone data and infinite-dimensional function spaces.

Purpose of the Study:

• To provide theoretical arguments for using Deep Neural Networks (DNNs) to approximate solutions of parametric PDEs, addressing the curse of dimensionality.
• To establish practical existence theorems for DNNs that overcome limitations of data scarcity and high dimensionality in approximating complex functions.
• To develop a theoretical framework for non-intrusive methods in high-dimensional approximation for CSE.

Main Methods:

• Establishing existence theorems for DNNs with dimension-independent architecture.
• Developing training procedures based on minimizing a regularized ℓ²-loss.
• Extending compressed sensing for Banach-valued vectors and polynomial emulation with DNNs.

Main Results:

• Demonstrating that DNNs can overcome the curse of dimensionality for approximating parametric PDE solutions.
• Achieving near-optimal algebraic convergence rates with respect to the amount of training data (m).
• Accounting for all error sources (sampling, optimization, approximation, discretization) in high-fidelity DNN approximations from coarse data.

Conclusions:

• Deep Neural Networks (DNNs) offer a theoretically sound, non-intrusive approach for high-dimensional approximation in Computational Science and Engineering.
• The developed methods provide a viable alternative to classical techniques, particularly when dealing with scarce data and complex, high-dimensional problems.
• This work bridges the gap between DL applications and rigorous numerical analysis, enhancing the reliability and applicability of DL in scientific computing.