Behavior of the Gibbs Sampler When Conditional Distributions Are Potentially Incompatible

Area of Science:

• Statistics
• Computational Statistics

Background:

• The Gibbs sampler is a standard algorithm in statistics, typically converging to a unique joint distribution.
• Its application is expanding to areas like multiple imputation, where conditional distributions may not be compatible.

Purpose of the Study:

• To investigate the behavior of the Gibbs sampler when conditional distributions are not compatible.
• To demonstrate that the convergence of the Gibbs sampler depends on the sampling order.

Main Methods:

• Mathematical analysis of the Gibbs sampler's convergence properties.
• Examination of the impact of sampling order on the resulting distribution.
• Illustrative examples to explain the observed behavior.

Main Results:

• The Gibbs sampler converges to different distributions based on the order of sampling when conditionals are incompatible.
• The convergence is not to a unique invariant distribution in such cases.

Conclusions:

• The order of sampling is a critical factor determining the outcome of a Gibbs sampler with incompatible conditionals.
• Understanding this dependency is crucial for applications like multiple imputation.