Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach

Area of Science:

• Machine Learning
• Statistical Modeling

Background:

• Multivariate Gaussian Processes (MGP) extend Gaussian Processes (GP) for multiple outputs.
• Convolution Processes (CP) are used to model commonalities among outputs in MGPs.
• Existing CP methods face challenges with computational demands and negative knowledge transfer.

Purpose of the Study:

• To address the computational challenges and negative transfer issues in MGP construction.
• To propose a novel regularized pairwise modeling approach for MGPs.
• To improve the scalability and accuracy of multi-output Gaussian process modeling.

Main Methods:

• Developed a regularized pairwise modeling approach for MGPs based on CP.
• Distributed the estimation of the full multivariate model into a group of bivariate GPs.
• Incorporated a penalty on latent functions to manage information sharing and prevent negative transfer.
• Combined predictions from bivariate models within a Bayesian framework for final predictions.

Main Results:

• The proposed method demonstrates excellent scalability for a large number of outputs.
• Effectively minimizes negative knowledge transfer between uncorrelated outputs.
• Statistical guarantees for the method were established.
• Numerical studies confirmed the advantageous features of the approach.

Conclusions:

• The regularized pairwise modeling approach offers an efficient and robust solution for MGP.
• It successfully tackles the computational complexity and negative transfer problems inherent in CP-based MGPs.
• The method provides a scalable and reliable framework for multi-output modeling with improved performance.