Efficient High-Dimensional Learning With Adaptive Gaussian RBF Networks

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Science

Background:

• Radial basis function neural networks (RBFNNs) offer rapid modeling and efficient learning.
• RBFNNs face challenges with high-dimensional data, including ineffective hidden layer activation and inefficient weight estimation.
• Existing methods struggle with numerical underflow and parameter tuning in high-dimensional spaces.

Purpose of the Study:

• To address the limitations of RBFNNs in high-dimensional data processing.
• To develop novel techniques for improved RBFNN performance and numerical stability.
• To enhance the efficiency of weight estimation in high-dimensional RBFNN models.

Main Methods:

• Proposed a dimensionality-adaptive Gaussian kernel function (DAGKF) with a novel width adjustment mechanism.
• Introduced a multioutput coordinate descent (MOCD) algorithm for parallel computation across multioutput systems.
• Developed the joint residual MOCD (JRMOCD) algorithm incorporating a joint residual criterion for effective weight estimation, with proven convergence.

Main Results:

• The DAGKF mitigates numerical difficulties in high-dimensional spaces.
• The MOCD and JRMOCD algorithms enable parallel computation and more effective weight estimation, avoiding simultaneous processing of entire feature matrices.
• Extensive experiments confirmed the superior performance of the proposed methods, especially in high-dimensional settings.

Conclusions:

• The developed DAGKF and JRMOCD algorithms significantly improve RBFNN performance for high-dimensional data.
• These methods offer robust solutions to numerical instability and computational inefficiency in RBFNNs.
• The findings pave the way for more effective application of RBFNNs in complex, high-dimensional machine learning tasks.