Sparsity-Aware Distributed Learning for Gaussian Processes With Linear Multiple Kernel.

Area of Science:

• Machine Learning
• Signal Processing
• Optimization

Background:

• Gaussian processes (GPs) are vital in machine learning and signal processing.
• Effective GP performance relies on kernel design and hyperparameter optimization.
• Existing methods face challenges with large-scale, multidimensional data and privacy concerns.

Purpose of the Study:

• To propose a novel Grid Spectral Mixture Product (GSMP) kernel for multidimensional data.
• To develop a sparsity-aware distributed learning framework (SLIM-KL) for hyperparameter optimization.
• To enhance prediction performance and efficiency while ensuring data privacy and minimizing communication costs.

Main Methods:

• Introduced the Grid Spectral Mixture Product (GSMP) kernel, reducing hyperparameters for multidimensional data.
• Developed the Sparse Linear Multiple Kernel Learning (SLIM-KL) framework for hyperparameter optimization.
• Utilized a quantized Alternating Direction Method of Multipliers (ADMMs) and Distributed Successive Convex Approximation (DSCA) for collaborative learning.

Main Results:

• The GSMP kernel demonstrated good approximation capabilities with reduced hyperparameters.
• Hyperparameter optimization of the GSMP kernel yielded sparse solutions.
• SLIM-KL framework effectively managed large-scale optimization, ensuring data privacy and low communication costs.

Conclusions:

• The proposed GSMP kernel and SLIM-KL framework offer superior prediction performance and efficiency.
• The methods are suitable for large-scale hyperparameter optimization of Gaussian processes.
• The distributed learning approach ensures data privacy and communication efficiency.