Gaussian kernel width optimization for sparse Bayesian learning.

Area of Science:

• Machine Learning
• Computational Statistics
• Pattern Recognition

Background:

• Sparse kernel methods are crucial for regression and classification tasks.
• Method performance and sparsity depend heavily on appropriate kernel function and parameter selection.
• Current parameter selection often relies on cross-validation, which can be computationally intensive.

Purpose of the Study:

• To present an extension of the relevance vector machine (RVM) capable of optimizing kernel parameters during training.
• To develop a method that reduces the dependency on the initial selection of kernel parameters.
• To maintain the computational efficiency and convergence speed of standard RVM.

Main Methods:

• Introduced a novel learning method extending the relevance vector machine (RVM).
• Employed an expectation-maximization (EM) algorithm for simultaneous updating of kernel and model parameters.
• Incorporated constraints during optimization to control the convergence of the fully parameterized model.

Main Results:

• The proposed method effectively finds optimal kernel parameters within the training procedure.
• Experimental results show comparable speed of convergence and computational complexity to standard RVM.
• Demonstrated reduced performance dependency on initial kernel parameter choices across synthetic and benchmark datasets.

Conclusions:

• The proposed RVM extension offers an effective solution for automatic kernel parameter optimization.
• This method enhances the robustness and reliability of sparse kernel methods in practical applications.
• The approach validates the effectiveness of expectation-maximization for kernel parameter tuning in machine learning.