A Geometric Variational Approach to Bayesian Inference.

Area of Science:

• Computational Statistics
• Bayesian Inference
• Differential Geometry

Background:

• Variational inference is a key technique for approximating posterior distributions in Bayesian models.
• Existing methods often rely on Kullback-Leibler divergence, which has limitations in providing tight bounds.
• The geometry of probability distributions offers potential for developing more effective inference methods.

Purpose of the Study:

• To develop a novel Riemannian geometric framework for variational inference in Bayesian models.
• To leverage the Fisher-Rao metric on the manifold of probability density functions.
• To introduce a new variational approach offering improved bounds compared to existing methods.

Main Methods:

• Utilizing a square-root density representation to map probability densities onto a hypersphere.
• Formulating variational inference as a problem on the hypersphere using α-divergence.
• Developing a novel gradient-based algorithm based on Fréchet derivatives for the variational problem.

Main Results:

• The proposed framework identifies the manifold of probability densities with a hypersphere, simplifying the geometry.
• The α-divergence formulation provides tighter lower bounds and novel upper bounds on marginal distributions.
• A new gradient-based algorithm demonstrates utility in simulations and real-world applications.

Conclusions:

• The Riemannian geometric framework offers a powerful new perspective for variational inference in Bayesian models.
• The proposed method enhances approximation accuracy and provides tighter bounds.
• The developed algorithm is effective and applicable to various Bayesian models.