Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and, crucially, sample complexity. For example, approximating solutions to parametric PDEs is a key task in UQ for CSE. Yet, training data for such problems is often scarce and corrupted by errors. Moreover, the target function, while often smooth, is a potentially infinite-dimensional function taking values in the PDE solution space, which is generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture widths and depths, and training procedures based on minimizing a (regularized) ℓ²-loss which achieve near-optimal algebraic rates of convergence in terms of the amount of training data m. These results involve key extensions of compressed sensing for recovering Banach-valued vectors and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.