An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

Area of Science:

• Computational statistics
• Bayesian inference
• Image processing

Background:

• Markov chain Monte Carlo (MCMC) algorithms are crucial for Bayesian inverse problems.
• High-dimensional parameter spaces and complex dependencies in Gaussian distributions pose significant challenges for MCMC performance.
• Designing efficient Metropolis-Hastings proposals that leverage local density geometry is computationally demanding.

Purpose of the Study:

• To address the challenges of sampling from high-dimensional Gaussian distributions with heterogeneous dependencies in Bayesian inverse problems.
• To improve the convergence and mixing properties of stochastic sampling algorithms.
• To reduce the computational cost of MCMC methods in complex statistical models.

Main Methods:

• Introduction of auxiliary variables to decouple heterogeneous sources of correlation in the model.
• Augmentation of the parameter space to isolate dependencies related to target parameters from those captured by auxiliary variables.
• Application of Gibbs sampling in the augmented space.

Main Results:

• The proposed method simplifies the sampling problem by reducing heterogeneous correlations in the conditional distribution.
• Significant reduction in the computational cost per Gibbs sampler iteration.
• Demonstrated good mixing properties in the parameter space.

Conclusions:

• Adding auxiliary variables effectively simplifies complex Bayesian inverse problems with Gaussian distributions.
• The approach enhances the efficiency and performance of MCMC algorithms in high-dimensional settings.
• The method shows practical utility in image restoration tasks.