Hamiltonian Monte Carlo acceleration using surrogate functions with random bases.

Area of Science:

• Computational statistics
• Bayesian inference
• Machine learning

Background:

• Bayesian methods are powerful but computationally intensive, limiting their use in big data analysis.
• Improving the computational efficiency of Bayesian inference is crucial for practical applications.
• Markov chain Monte Carlo (MCMC) methods, including Hamiltonian Monte Carlo (HMC), are widely used but can be slow.

Purpose of the Study:

• To propose an efficient and scalable computational technique for Hamiltonian Monte Carlo (HMC).
• To address the high computational cost associated with Bayesian inference in big data settings.
• To develop a flexible sampling algorithm that converges to the correct target distribution.

Main Methods:

• Exploiting parameter space structure and regularity to approximate geometric properties.
• Constructing a surrogate function using random bases and an optimization process to approximate the target distribution.
• Relating the method to generalized additive models and Gaussian process models through basis function and optimization choices.

Main Results:

• The proposed method provides a flexible, scalable, and efficient sampling algorithm.
• Experiments on simulated and real data demonstrate substantially improved sampling efficiency compared to state-of-the-art methods.
• The technique converges correctly to the target distribution.

Conclusions:

• The novel computational technique significantly enhances the efficiency of Hamiltonian Monte Carlo for big data analysis.
• This approach offers a scalable and flexible solution for complex Bayesian inference problems.
• The method shows promise for broader adoption of Bayesian techniques in data-intensive fields.