Scaling Multidimensional Inference for Structured Gaussian Processes.

Area of Science:

• Machine Learning
• Statistical Modeling
• Computational Statistics

Background:

• Exact Gaussian Process (GP) regression is computationally intensive, with O(N^3) runtime, limiting its application to large datasets.
• Existing GP scaling improvements often involve covariance matrix approximation or exploiting specific data structures like Markov properties or lattice inputs.
• These scalable GP methods have not been effectively extended to multidimensional input spaces, which are common in many applications.

Purpose of the Study:

• To develop and evaluate novel extensions of structured Gaussian Processes for multidimensional input settings.
• To address the computational intractability of exact GP regression for large N.
• To adapt existing structured GP techniques for both additive and multiplicative kernel structures in higher dimensions.

Main Methods:

• Introduced a new inference method for additive GPs, linking backfitting with Bayesian inference.
• Extended additive GPs using projection pursuit regression and Laplace approximation for non-Gaussian observations.
• Developed a novel method for GPs with multiplicative kernel structure on multidimensional grids.

Main Results:

• The proposed methods demonstrate significant computational savings compared to naive GP regression.
• Performance achieved is comparable or very close to exact GP methods.
• Successfully applied the novel extensions to several diverse datasets.

Conclusions:

• The novel structured GP extensions effectively handle multidimensional inputs and large datasets.
• These methods offer a computationally efficient alternative to exact GP regression while maintaining high accuracy.
• The work bridges a gap in applying structured GPs to complex, multidimensional problems.