Deep Latent-Variable Kernel Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence

Background:

• Deep kernel learning (DKL) combines Gaussian processes (GP) and neural networks (NNs) for hybrid modeling.
• DKL offers representation learning and automatic regularization but faces challenges with deterministic encoders in NNs, especially on small datasets.

Purpose of the Study:

• Introduce a deep latent-variable kernel learning (DLVKL) model with stochastic encoding for improved regularization.
• Enhance DLVKL using neural stochastic differential equations (NSDE) for variational posteriors and a hybrid prior.

Main Methods:

• Developed a DLVKL model with stochastic latent variables for regularized representation.
• Integrated NSDE for expressive variational posteriors and a hybrid prior for flexible tradeoff control.

Main Results:

• DLVKL-NSDE demonstrates performance comparable to well-calibrated GPs on small datasets.
• The proposed model shows superior performance on large datasets compared to existing methods.

Conclusions:

• DLVKL-NSDE effectively addresses the regularization limitations of DKL models.
• The model offers a robust approach for both small and large-scale machine learning tasks.