Scalable Gaussian Process Classification With Additive Noise for Non-Gaussian Likelihoods.

Area of Science:

• Machine Learning
• Statistical Modeling
• Computational Statistics

Background:

• Gaussian Process Classification (GPC) offers a flexible framework for function space distributions but faces scalability and inference challenges with big data and non-Gaussian likelihoods.
• Existing scalable GPCs often rely on sparse approximations and analytical Evidence Lower Bounds (ELBO), but these methods are restricted to specific likelihoods or assumptions.

Purpose of the Study:

• To develop a unifying framework for scalable Gaussian process classification (GPC) that accommodates a wider range of likelihood functions.
• To address the limitations of conventional GPCs in terms of scalability and intractable inference for big data problems.

Main Methods:

• Introduced additive noises to augment the probability space, enabling unified treatment of various likelihoods (step, probit, logit, softmax) within a scalable GPC framework.
• Employed variational inference to derive analytical Evidence Lower Bounds (ELBO) for the proposed scalable GPC models.
• Developed a novel approach for GPC with softmax likelihood by utilizing noise variables themselves.

Main Results:

• The proposed framework successfully unifies scalable GPCs for multiple likelihood functions, including step, probit, logit, and softmax.
• Achieved analytical ELBO through variational inference, overcoming limitations of previous scalable GPC methods.
• Demonstrated superior empirical performance on extensive binary and multiclass classification tasks, handling datasets up to two million data points.

Conclusions:

• The presented unifying framework significantly enhances the applicability and efficiency of scalable Gaussian process classification.
• The method provides a robust and versatile approach for big data classification problems with diverse likelihoods.
• This work advances scalable GPC by enabling analytical inference across a broader spectrum of classification tasks.