Neural Operator Variational Inference Based on Regularized Stein Discrepancy for Deep Gaussian Processes

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Statistics

Background:

• Deep Gaussian Process (DGP) models are powerful for Bayesian inference but face intractability issues.
• Existing approximations like mean-field assumptions limit model expressiveness, while stochastic methods are computationally expensive.

Purpose of the Study:

• To introduce Neural Operator Variational Inference (NOVI) for efficient and accurate inference in Deep Gaussian Processes.
• To address the limitations of current approximation techniques in DGP models.

Main Methods:

• NOVI employs a neural generator for sampling and minimizes the regularized Stein discrepancy (RSD) in L2 space.
• The minimax problem is solved using Monte Carlo estimation and subsampling stochastic optimization.
• Bias control is achieved by adjusting Fisher divergence, ensuring algorithm stability and precision.

Main Results:

• The proposed NOVI method demonstrates effectiveness and faster convergence on datasets of varying sizes.
• Achieved 93.56% classification accuracy on CIFAR10, surpassing state-of-the-art Gaussian Process methods.
• Robust error control and algorithmic stability were validated through experiments.

Conclusions:

• NOVI offers a promising approach for enhancing deep Bayesian nonparametric models.
• The method has the potential for significant implications in various practical applications requiring accurate Bayesian inference.