Pseudo-Marginal Bayesian Inference for Gaussian Processes.

Area of Science:

• Statistics
• Machine Learning
• Probabilistic Modeling

Background:

• Gaussian process (GP) priors are powerful in probabilistic modeling but present challenges in exact Bayesian inference and uncertainty quantification for predictions.
• Existing methods struggle with efficient posterior simulation and integrating all model parameters, limiting full Bayesian analysis.

Purpose of the Study:

• To present a general and effective methodology for addressing challenges in Gaussian process prior adoption.
• To improve Bayesian inference and uncertainty quantification in GP-based hierarchical statistical models.

Main Methods:

• Utilized the pseudo-marginal approach to Markov chain Monte Carlo (MCMC).
• Applied probit regression as a working example to demonstrate the methodology.
• Developed improved sampling methods for the posterior distribution of Gaussian Process covariance function parameters.

Main Results:

• The proposed pseudo-marginal MCMC approach efficiently handles Bayesian inference and uncertainty quantification.
• Demonstrated superior performance over existing sampling methods for simulating posterior distributions.
• Showcased the feasibility of Monte Carlo integration for all model parameters, enhancing prediction uncertainty quantification.

Conclusions:

• The methodology provides a powerful tool for full Bayesian inference in GP-based hierarchical models.
• Offers superior quantification of uncertainty in predictions compared to state-of-the-art methods.
• Confirms the effectiveness through extensive comparisons with probabilistic classifiers.