LS-SVR as a Bayesian RBF Network.

Area of Science:

• Machine Learning
• Statistical Modeling
• Artificial Intelligence

Background:

• Least Squares Support Vector Regression (LS-SVR) is a popular machine learning technique.
• Bayesian Radial Basis Function (RBF) networks offer a probabilistic approach to regression.
• Existing literature suggests potential links between these methods, but a formal connection is lacking.

Purpose of the Study:

• To explicitly and formally establish theoretical similarities between LS-SVR with an RBF kernel and Maximum a Posteriori (MAP) inference on Bayesian RBF networks.
• To demonstrate these correspondences through empirical computational experiments.
• To explore the potential for enhancing LS-SVR by leveraging Bayesian methodologies.

Main Methods:

• Theoretical analysis to identify and formalize correspondences between LS-SVR and Bayesian RBF networks.
• Implementation of LS-SVR with an RBF kernel.
• Application of MAP inference on Bayesian RBF networks with a Gaussian prior on regression weights.
• Empirical validation using standard regression benchmarks.

Main Results:

• Formal theoretical similarities are established between LS-SVR (RBF kernel) and MAP inference on Bayesian RBF networks.
• Computational experiments confirm the theoretical findings on standard regression tasks.
• The study demonstrates a clear link between the two distinct learning paradigms.

Conclusions:

• The established theoretical link provides a foundation for improving LS-SVR.
• Leveraging Bayesian methodology offers new avenues for enhancing LS-SVR performance.
• This work bridges the gap between two significant machine learning approaches, opening new research directions.