Related Experiment Videos
Constructing Bayesian formulations of sparse kernel learning methods
Gavin C Cawley1, Nicola L C Talbot
1School of Computing Sciences, University of East Anglia, Norwich NR4 7TJ, UK. gcc@cmp.uea.ac.uk
Summary
This study introduces a simplified Bayesian approach for sparse kernel learning using incomplete Cholesky factorization. This method efficiently creates sparse models and simplifies Bayesian analysis for algorithms like kernel ridge regression.
Area of Science:
- Machine Learning
- Statistical Modeling
- Computational Statistics
Background:
- Bayesian inference in kernel learning often involves complex computations.
- Sparse kernel methods aim to improve efficiency and scalability.
- Existing methods may lack a unified framework for Bayesian treatment.
Purpose of the Study:
- To present a simplified technique for Bayesian analysis of sparse kernel learning algorithms.
- To enable efficient Bayesian inference using a unified framework.
- To demonstrate the method's applicability to kernel ridge regression and kernel logistic regression.
Main Methods:
- Employing incomplete Cholesky factorization to modify the dual parameter space.
- Whitening the Gaussian prior over dual model parameters.
- Utilizing MacKay's evidence framework for Bayesian analysis.
- Identifying a subset of training data as an approximate basis for sparsity.
Main Results:
- The regularisation term simplifies to a standard weight-decay regulariser.
- The incomplete Cholesky factorization naturally yields a sparse model by selecting a data subset.
- Demonstrated successful Bayesian treatments for kernel ridge regression (KRR) and kernel logistic regression (KLR).
Conclusions:
- The proposed technique offers a simplified and unified approach to Bayesian sparse kernel learning.
- The method enhances computational efficiency and model interpretability.
- The approach is expected to be widely applicable to various kernel learning algorithms.