Approximation bounds for smooth functions in C(IRd) by neural and mixture networks

Area of Science:

• Computational Mathematics
• Machine Learning Theory
• Numerical Analysis

Background:

• Feedforward neural networks are powerful function approximators.
• Approximation theory analyzes the error in approximating functions.
• Previous research focused on compact domains.

Purpose of the Study:

• To establish approximation bounds for smooth multivariate functions over IRd using single-hidden-layer neural networks.
• To generalize existing approximation results from compact domains to the entire real space.
• To extend these findings to mixture of expert architectures.

Main Methods:

• Utilizing nonlinear ridge functions in a single hidden layer of feedforward neural networks.
• Analyzing the smoothness of target functions and activation functions.
• Deriving upper bounds on the approximation degree over IRd.

Main Results:

• Established upper bounds for approximating smooth multivariate functions in C(IRd).
• Generalized approximation results beyond compact domains to the entire real space.
• Demonstrated that mixture of expert architectures achieve similar approximation bounds.

Conclusions:

• Single-hidden-layer neural networks with nonlinear ridge functions effectively approximate smooth functions over IRd.
• The theoretical framework is extended to more complex architectures like mixture of experts.
• This work advances the understanding of neural network approximation capabilities in unbounded domains.