Robust Gaussian Process Regression With Input Uncertainty: A PAC-Bayes Perspective.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Standard Gaussian Processes (GPs) require precise data and lack robust generalization guarantees.
• Input perturbations can cause regression model mismatching in traditional GPs.

Purpose of the Study:

• To propose a novel robust noisy input GP (NIGP) algorithm for handling data uncertainty.
• To provide numerical generalization performance guarantees on unknown data distributions using PAC Bayes theory.
• To develop sparse NIGP and PAC-Bayes NIGP for reduced computational complexity.

Main Methods:

• Development of a robust noisy input GP (NIGP) algorithm based on Probably Approximately Correct (PAC) Bayes theory.
• Introduction of a sparse NIGP algorithm to mitigate computational demands.
• Formulation of a sparse PAC-Bayes NIGP approach optimizing PAC-Bayes bounds for tighter generalization error bounds.

Main Results:

• NIGP algorithms demonstrate improved accuracy compared to standard methods.
• The proposed PAC-NIGP algorithms achieve robust performance with uncertain input and output data.
• Optimization of PAC-Bayes bounds offers a tighter generalization error upper bound than maximizing marginal log likelihood.

Conclusions:

• The novel PAC-NIGP algorithms effectively address data uncertainty in Gaussian processes.
• These methods offer enhanced robustness and superior generalization error bounds.
• The sparse PAC-NIGP approach provides a computationally efficient solution for complex datasets.