Improving sampling efficacy on high-dimensional distributions with thin high-density regions using Conservative...

Hamiltonian Monte Carlo is a prominent Markov Chain Monte Carlo algorithm, which employs symplectic integrators to sample from high-dimensional target distributions in many applications, such as statistical mechanics, Bayesian statistics, and generative models. However, such distributions tend to have thin high-density regions, posing a significant challenge for symplectic integrators to maintain the small energy errors needed for a high acceptance probability. Instead, we propose a variant called Conservative Hamiltonian Monte Carlo, using R-reversible energy-preserving integrators to retain a high acceptance probability. We show our algorithm can achieve approximate stationarity with an error determined by the Jacobian approximation of the energy-preserving proposal map and its approximation error due to inexact solutions of implicit schemes. Numerical evidence shows improved convergence and robustness over integration parameters on target distributions with thin high-density regions and in high dimensions. Moreover, a version of our algorithm can also be applied to target distributions without gradient information.