Laplace Approximation for Divisive Gaussian Processes for Nonstationary Regression.

Area of Science:

• Machine Learning
• Statistical Modeling
• Computational Statistics

Background:

• Standard Gaussian Process (GP) regression relies on restrictive stationary assumptions (constant noise power, covariance dependent only on input differences).
• These assumptions are often unrealistic for real-world data, limiting GP applicability.
• Existing methods for nonstationarity require prior knowledge of the nonstationarity type, which is frequently unavailable.

Purpose of the Study:

• To develop a novel divisive GP model for nonstationary regression.
• To incorporate heteroscedastic noise handling within the GP framework.
• To achieve accurate inference with reduced computational cost compared to existing methods.

Main Methods:

• Utilized Laplace approximation for inference within a divisive GP model.
• Leveraged the log-concavity of the likelihood to ensure a unimodal posterior and guarantee convergence of the Laplace approximation.
• Compared performance against Expectation Propagation (EP) and Markov Chain Monte Carlo (MCMC) with Elliptical Slice Sampling (ESS).

Main Results:

• The proposed Laplace approximation method effectively performs nonstationary regression, including cases with heteroscedastic noise.
• Achieved accurate posterior approximations comparable to EP and ESS.
• Demonstrated a significantly reduced computational load compared to both EP and ESS.

Conclusions:

• The divisive GP model with Laplace approximation offers an effective and computationally efficient solution for nonstationary regression problems.
• This approach relaxes restrictive stationary assumptions without requiring prior knowledge of nonstationarity.
• The method provides a practical alternative for handling complex real-world data exhibiting nonstationary characteristics and heteroscedastic noise.