Divisive Gaussian processes for nonstationary regression | JoVE Visualize

Area of Science:

Machine Learning
Statistical Modeling

Background:

Standard Gaussian Process Regression (GPR) often assumes constant noise power (homoscedasticity) and stationary covariance functions, which are restrictive for real-world data.
Achieving nonstationarity typically requires specific covariance functions, but prior knowledge about such nonstationarity is often unavailable.
The homoscedastic assumption, while simplifying inference, limits the applicability of GPR to problems with varying noise levels.

Purpose of the Study:

To develop a flexible Gaussian process regression (GPR) model capable of handling nonstationary data with heteroscedastic noise.
To enable accurate and computationally efficient inference for this new model, overcoming analytical intractability.
To provide a robust alternative to standard GPR for complex, real-world regression tasks.

Main Methods:

A novel divisive GPR model is proposed, utilizing the pointwise division of two nonparametric latent functions to model nonstationarity and heteroscedasticity.
Variational posterior approximation via expectation propagation (EP) is introduced to facilitate tractable and accurate inference.
A Markov chain Monte Carlo (MCMC) implementation using elliptical slice sampling is developed to validate the accuracy of the EP approximation.

Main Results:

The proposed divisive GPR model effectively performs nonstationary regression under heteroscedastic noise conditions.
The expectation propagation (EP) approximation provides accurate inference at a reduced computational cost compared to exact methods.
Experimental results demonstrate the practical utility and effectiveness of the developed approach.

Conclusions:

The divisive GPR model offers a powerful framework for addressing complex regression problems with nonstationary and heteroscedastic characteristics.
The proposed variational inference method enables efficient and reliable application of the model.
This approach expands the capabilities of Gaussian process regression for diverse scientific and engineering applications.