Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling

Area of Science:

• Statistical modeling
• Information theory
• Computational neuroscience

Background:

• Markov chains are established tools for analyzing sequential data in time or space.
• Existing methods often focus on lower-order chains or specific data types.

Purpose of the Study:

• To develop a Bayesian framework for inferring kth order Markov chains from finite data.
• To connect Bayesian inference with statistical mechanics and information theory.
• To introduce methods for model-order selection and inferring out-of-class processes.

Main Methods:

• Bayesian parameter estimation and model-order selection for kth order Markov chains.
• Information-theoretic (type theory) techniques linking Bayesian evidence to partition functions.
• Finite data-size scaling with model-order comparison.

Main Results:

• Established a direct relationship between Bayesian evidence and the partition function.
• Enabled straightforward calculation of conditional relative entropy and source entropy rate expectations and variances.
• Introduced a novel method for inferring the structure of out-of-class processes.

Conclusions:

• The proposed Bayesian approach provides a robust framework for inferring complex Markov chain models.
• The connection to statistical mechanics offers new insights into information-theoretic properties of these models.
• The developed methods enhance the ability to analyze and understand complex sequential data.