Conditional Deep Gaussian Processes: Multi-Fidelity Kernel Learning.

Area of Science:

• Machine Learning
• Bayesian Inference
• Statistical Modeling

Background:

• Deep Gaussian Processes (DGPs) provide expressive Bayesian modeling with inherent uncertainty quantification.
• DGPs' hierarchical structure is suitable for multi-fidelity regression tasks involving sparse high-precision and abundant low-precision data.

Purpose of the Study:

• To propose a conditional Deep Gaussian Process (DGP) model that directly utilizes lower-fidelity data.
• To enhance multi-fidelity regression by incorporating information from varied data precision levels.

Main Methods:

• Developed a conditional DGP model where latent Gaussian Processes (GPs) are supported by fixed lower-fidelity data.
• Applied moment matching to approximate the marginal prior of the conditional DGP with a GP.
• Learned hyperparameters by optimizing the approximate marginal likelihood.

Main Results:

• The conditional DGP model achieved performance comparable to existing multi-fidelity regression methods, variational inference, and multi-output GPs.
• Effective kernels derived from the model implicitly incorporated lower-fidelity data, demonstrating the benefits of hierarchical distribution propagation.
• Experiments on synthetic and high-dimensional datasets validated the model's efficacy.

Conclusions:

• The proposed conditional DGP model effectively integrates multi-fidelity data through its hierarchical structure.
• The resulting effective kernels encode inductive biases beneficial for accurate function approximation.
• This approach offers a powerful method for leveraging diverse data sources in regression problems.