Comparing fixed and variable-width Gaussian networks.

Area of Science:

• Computational mathematics
• Machine learning theory

Background:

• Gaussian radial-basis-functions (RBFs) are fundamental in computational models.
• Understanding the impact of Gaussian width is crucial for network design and analysis.

Purpose of the Study:

• To investigate the role of Gaussian width in RBF networks and Gaussian kernel networks.
• To explore the effect of width on functional equivalence and universal approximation properties.
• To analyze norms in reproducing kernel Hilbert spaces (RKHS) concerning Gaussian width.

Main Methods:

• Comparative analysis of two computational models: varying-width/center RBFs and fixed-width/varying-center Gaussian kernel networks.
• Theoretical exploration of functional equivalence, universal approximation, and RKHS norms.
• Mathematical proofs regarding network unit properties and function set characteristics.

Main Results:

• Functional equivalence in Gaussian RBF networks necessitates identical numbers of units, centers, and widths.
• Sets of functions generated by Gaussian kernel networks with different widths are disjoint but individually capable of universal approximation.
• Embedding of RKHSs induced by flatter Gaussians into those induced by sharper Gaussians is described, with norm ratio growth estimated against input dimension.

Conclusions:

• Gaussian width is a critical parameter influencing the functional properties and representational capacity of neural networks.
• The study provides theoretical insights into the structure and capabilities of RBF and Gaussian kernel networks.
• Findings contribute to a deeper understanding of universal approximation and function space geometry in machine learning.