Improving sampling efficacy on high-dimensional distributions with thin high-density regions using Conservative...

Area of Science:

• Computational Statistics
• Statistical Mechanics
• Machine Learning

Background:

• Hamiltonian Monte Carlo (HMC) is vital for sampling high-dimensional distributions in Bayesian statistics and generative models.
• Symplectic integrators in HMC struggle with thin high-density regions, leading to low acceptance probabilities due to energy errors.

Purpose of the Study:

• To introduce Conservative Hamiltonian Monte Carlo (CHMC), a novel variant designed to enhance sampling efficiency.
• To address the limitations of standard HMC in handling distributions with challenging characteristics.

Main Methods:

• Proposing CHMC, which utilizes R-reversible energy-preserving integrators.
• Analyzing the approximate stationarity error based on Jacobian approximation and inexact implicit scheme solutions.

Main Results:

• CHMC demonstrates improved convergence and robustness across various integration parameters.
• The algorithm shows superior performance on high-dimensional target distributions with thin high-density regions.

Conclusions:

• CHMC offers a more effective approach for sampling complex distributions compared to standard HMC.
• A variant of CHMC can even be applied to distributions lacking gradient information, expanding its applicability.