Adaptive Incremental Mixture Markov Chain Monte Carlo.

Area of Science:

• Computational Statistics
• Bayesian Inference
• Machine Learning

Background:

• Markov chain Monte Carlo (MCMC) methods are crucial for Bayesian inference.
• Adaptive MCMC methods often use global updates, which can be inefficient for complex distributions.
• Existing methods struggle with high-dimensional and multimodal target distributions.

Purpose of the Study:

• To propose a novel adaptive MCMC algorithm, AIMM, for sampling from challenging probability distributions.
• To develop a method that locally adapts a semiparametric proposal kernel.
• To address limitations of global adaptation rules in existing adaptive MCMC techniques.

Main Methods:

• AIMM utilizes an independent Metropolis-Hastings proposal based on a finite mixture of Gaussian distributions.
• The algorithm locally adapts by adding mixture components when discrepancies with the target distribution are detected.
• The number of mixture components is not fixed a priori, allowing for dynamic adaptation.

Main Results:

• Theoretical guarantees show a related stochastic process converges to the correct target distribution.
• Empirical results demonstrate AIMM's effectiveness on high-dimensional and multimodal distributions.
• Successful application to Bayesian inference for a semiparametric regression model on the Boston Housing dataset.

Conclusions:

• AIMM offers a flexible and effective approach for sampling from difficult probability distributions.
• The local adaptation strategy enhances performance in complex scenarios.
• The method shows practical utility in real-world Bayesian inference problems.