Exact Gaussian processes for massive datasets via non-stationary sparsity-discovering kernels.

Area of Science:

• Computational Mathematics
• Machine Learning
• Scientific Computing

Background:

• Gaussian Processes (GPs) are powerful for stochastic function approximation, offering analytical tractability, robustness, and uncertainty quantification.
• Exact GPs face significant computational and storage challenges ([Formula: see text] and [Formula: see text] complexity) with large datasets.
• Existing approximate methods often compromise accuracy and kernel design flexibility.

Purpose of the Study:

• To develop a method for scaling exact Gaussian Processes to large datasets without compromising accuracy.
• To leverage naturally occurring sparsity in data and physical processes for efficient GP computation.
• To enable the design of flexible, expressive kernels that can learn sparse structures.

Main Methods:

• Proposed a novel approach where kernels discover, rather than induce, sparse structure.
• Developed ultra-flexible, compactly-supported, and non-stationary kernels capable of learning zero covariances.
• Combined kernel design with High-Performance Computing (HPC) and constrained optimization techniques.

Main Results:

• Achieved scalability for exact Gaussian Processes to datasets exceeding 5 million data points.
• Maintained analytical tractability and uncertainty quantification inherent to exact GPs.
• Enabled greater flexibility in kernel design for modeling complex physical processes.

Conclusions:

• This new method overcomes the computational limitations of exact GPs for large-scale applications.
• The ability of kernels to learn sparsity offers a more principled and accurate approach to GP approximation.
• This advancement significantly expands the applicability of exact Gaussian Processes in science and engineering.