Radial Basis Function Networks: Generalization in Over-realizable and Unrealizable Scenarios

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Radial basis function (RBF) networks are key in machine learning.
• Understanding their generalization error is crucial for model performance.
• Stochastic training paradigms and Bayesian methods offer powerful analytical tools.

Purpose of the Study:

• To analyze learning and generalization in two-layer RBF networks under stochastic training.
• To derive expressions for generalization error using a Bayesian approach.
• To investigate the impact of regularization (weight decay) and network realizability.

Main Methods:

• Utilized a Bayesian framework to derive generalization error expressions.
• Assumed the data-generating mechanism is an RBF network (teacher model).
• Examined both over-realizable (student > teacher) and unrealizable (teacher > student) cases.

Main Results:

• Derived analytic expressions for generalization error in RBF networks.
• Quantified the effects of weight decay regularization.
• Demonstrated that dependence on teacher network centers can be removed via a confidence parameter.

Conclusions:

• The study provides a theoretical framework for understanding RBF network generalization.
• Analytic results are validated through simulations, confirming the derived expressions.
• The findings contribute to the design and training of more effective RBF networks.