On the construction of the relevance vector machine based on Bayesian Ying-Yang harmony learning

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Tipping's Relevance Vector Machine (RVM) utilizes kernel methods for sparse basis function networks, outperforming Support Vector Machines (SVMs) in sparsity and hyperparameter estimation.
• Original RVM performance is constrained by prior smoothness assumptions, potentially causing underfitting or overfitting due to kernel-dependent sparsity control.

Purpose of the Study:

• To enhance RVM performance by explicitly incorporating the number of basis functions into optimization.
• To develop a novel RVM construction method using Bayesian Ying-Yang (BYY) harmony learning.

Main Methods:

• The proposed method integrates the number of basis functions directly into the optimization objective.
• It employs Xu's Bayesian Ying-Yang (BYY) harmony learning to maximize agreement between forward training and backward testing probability distributions.

Main Results:

• The novel RVM methodology achieved minimal structural complexity.
• The approach demonstrated superior goodness of fit to the data compared to conventional RVMs.

Conclusions:

• The BYY harmony learning technique effectively addresses RVM limitations related to prior assumptions and kernel parameter sensitivity.
• This research presents a robust RVM framework balancing model complexity and predictive accuracy.