Variational Hybrid Monte Carlo for Efficient Multi-Modal Data Sampling.

Area of Science:

• Computational Statistics
• Machine Learning Algorithms
• Probabilistic Modeling

Background:

• Hamiltonian Monte Carlo (HMC) is an efficient Markov Chain Monte Carlo (MCMC) method leveraging Hamiltonian dynamics.
• HMC offers superior exploration over random-walk methods but can exhibit high sample autocorrelation.
• Sampling from multi-modal distributions poses challenges for dynamics-based MCMC, risking entrapment in local modes.

Purpose of the Study:

• To reduce sample autocorrelation in HMC.
• To develop an MCMC method capable of effectively sampling from multi-modal distributions.
• To provide theoretical guarantees for the convergence of the proposed methods.

Main Methods:

• Proposed Langevin Hamiltonian Monte Carlo (LHMC) to decrease sample autocorrelation.
• Introduced Variational Hybrid Monte Carlo (VHMC) utilizing variational distributions for phase space exploration.
• Developed a method to effectively sample from challenging multi-modal distributions.

Main Results:

• LHMC effectively reduces autocorrelation in HMC samples.
• VHMC demonstrates superior performance in sampling from multi-modal distributions.
• Theoretical convergence guarantees are formally proven for the proposed methods.

Conclusions:

• The proposed LHMC and VHMC methods enhance MCMC sampling efficiency and robustness.
• VHMC successfully addresses the challenge of sampling from multi-modal distributions.
• Experimental results validate the theoretical findings and demonstrate practical effectiveness.