Asynchronous Parallel Large-Scale Gaussian Process Regression.

Area of Science:

• Machine Learning
• Statistical Modeling

Background:

• Gaussian Process Regression (GPR) is a powerful nonparametric method with diverse applications.
• Training large-scale GPR models is computationally intensive and memory-demanding.

Purpose of the Study:

• To develop an efficient algorithm for large-scale Gaussian Process Regression training.
• To address the computational and memory challenges associated with large GPR models.

Main Methods:

• Formulated GPR as a convex optimization problem (kernel ridge regression).
• Employed random feature mapping for kernel approximation.
• Utilized asynchronous doubly stochastic gradient with variance reduction and coordinate descent for parallel updates.

Main Results:

• The proposed algorithm demonstrates scalability in both sample size and dimensionality.
• Achieved significant speed-up in training computation.
• Proven global linear convergence rate.
• Experimental results show superior performance over state-of-the-art GPR methods on benchmark datasets.

Conclusions:

• The asynchronous doubly stochastic gradient algorithm effectively handles large-scale GPR.
• Offers a computationally efficient and scalable solution for GPR training.
• Provides a promising alternative to existing GPR methods for complex datasets.