Bayesian inference based on stationary Fokker-Planck sampling.

Area of Science:

• Computational statistics
• Machine learning
• Bayesian inference

Background:

• Complex inference models pose challenges for traditional Bayesian learning.
• Existing sampling methods like Gibbs sampling have limitations with arbitrary densities.

Purpose of the Study:

• To introduce a novel formalism for Bayesian learning using stationary Fokker-Planck (SFP) sampling.
• To demonstrate the efficacy of SFP in handling complex inference models and arbitrary posterior densities.

Main Methods:

• Developed a novel formalism based on the stationary Fokker-Planck (SFP) approach for sampling from posterior densities.
• Generalized the Gibbs sampler for arbitrary and unknown conditional densities using SFP.
• Constructed approximate analytical expressions for posterior conditionals and marginals.
• Outlined efficient learning methods for artificial neural networks utilizing analytical marginals.

Main Results:

• SFP sampling converges to the full joint posterior using approximate conditionals within a Gibbs sampling process.
• Efficient offline and incremental Bayesian inference and maximum likelihood estimation were achieved in classification and regression.
• SFP demonstrated the ability to traverse large low-probability regions without step-size parameter tuning.
• Computation cost of SFP grows linearly with model dimension.

Conclusions:

• The proposed stationary Fokker-Planck (SFP) sampling offers a powerful and efficient method for Bayesian learning in complex models.
• SFP provides a robust alternative to existing Monte Carlo strategies, requiring minimal parameter tuning.
• The method facilitates efficient learning in artificial neural networks and various inference tasks.