Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Variational inference approximates complex probability distributions using simpler variational families.
• Gaussian variational families are standard due to ease of sampling and computation.
• Approximating intractable posteriors remains a key challenge in Bayesian inference.

Purpose of the Study:

• To develop a novel, efficient algorithm for Gaussian variational approximation.
• To analyze the convergence properties of the proposed method.
• To demonstrate the algorithm's performance on high-dimensional problems.

Main Methods:

• Viewing variational approximation through the lens of gradient flows.
• Developing a particle-based approximation algorithm based on a linear flow.
• Theoretical analysis of linear convergence for Gaussian targets.

Main Results:

• The proposed algorithm achieves linear convergence to the exact solution for Gaussian targets with sufficient particles.
• It provides a low-rank approximation for non-Gaussian targets.
• Empirical results show outperformance on Gaussian targets and competitive performance on non-Gaussian targets in high-dimensional settings.

Conclusions:

• The novel gradient flow-based particle method offers an efficient and effective alternative for variational inference.
• The algorithm's linear convergence guarantees and strong empirical performance make it suitable for complex Bayesian models.
• This approach advances the field of approximate Bayesian computation.