GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework.

Area of Science:

• Machine Learning
• Computational Statistics
• Bayesian Inference

Background:

• Particle-based Variational Inference (ParVI) methods are crucial for deep Bayesian inference, efficiently generating samples using target distribution scores.
• Existing ParVI methods approximate Wasserstein gradient flow for particle system refinement, with recent advances focusing on accelerated updates or dynamic weight adjustments.

Purpose of the Study:

• Introduce a novel semi-Hamiltonian gradient flow (SHIFR flow) in Information-Fisher-Rao space.
• Propose the General Accelerated Dynamic-Weight Particle-based Variational Inference (GAD-PVI) framework, the first to simultaneously achieve accelerated position updates and dynamic weight adjustments.

Main Methods:

• Developed the SHIFR flow operating in a novel Information-Fisher-Rao space.
• Formulated the GAD-PVI framework, which integrates accelerated position updates and dynamic weight adjustments.
• Ensured GAD-PVI's compatibility with various dissimilarities and gradient flow approximation methods.

Main Results:

• GAD-PVI demonstrated faster convergence compared to state-of-the-art methods in experimental evaluations.
• The proposed framework achieved reduced approximation errors in both score-based and sample-based Bayesian inference tasks.
• GAD-PVI successfully generated high-quality samples even without analytical scores when appropriate dissimilarities were employed.

Conclusions:

• The GAD-PVI framework represents a significant advancement in particle-based variational inference.
• It offers simultaneous acceleration and dynamic weight adaptation, outperforming existing methods in efficiency and accuracy.
• GAD-PVI broadens the applicability of ParVI, particularly in scenarios where analytical scores are unavailable.