High-Dimensional Function Approximation With Neural Networks for Large Volumes of Data.

Area of Science:

• Machine Learning
• Numerical Analysis
• Data Science

Background:

• High-dimensional function approximation using neural networks faces challenges due to the curse of dimensionality.
• Data often resides on lower-dimensional manifolds, suggesting that exploiting this structure can improve approximation.
• Existing methods show promise in projecting data manifolds for improved neural network approximation.

Purpose of the Study:

• To investigate the approximation error of neural networks trained on projected data spaces.
• To determine if manifold projection enhances neural network approximation performance despite data sparsity.
• To identify optimal sampling strategies for generating low-dimensional projections.

Main Methods:

• Analyzing approximation errors of neural networks trained in projected lower-dimensional spaces.
• Comparing performance against neural networks trained directly on high-dimensional data.
• Evaluating the impact of sparse data sampling on projection quality and approximation accuracy.
• Investigating the benefits of uniformly distributed sparse samples for projection generation.

Main Results:

• Neural networks trained on projected data spaces exhibit superior approximation performance compared to those trained on high-dimensional data.
• This improvement holds even when the projection is based on a relatively sparse data sample.
• Uniformly distributed sparse samples are preferable for generating effective low-dimensional projections.

Conclusions:

• Leveraging low-dimensional manifold structure through projection is a viable strategy to mitigate the curse of dimensionality in neural network function approximation.
• Effective approximation is achievable even with limited data, provided the projection is well-generated.
• The findings have practical implications for applying neural networks to complex, high-dimensional datasets.