Reduction of Markov Chains Using a Value-of-Information-Based Approach

Area of Science:

• Information Theory
• Markov Chain Analysis
• Model Reduction

Background:

• Markov chains are widely used for modeling complex systems.
• High-order Markov chains can become computationally intractable due to large state spaces.
• Existing model reduction techniques may require prior knowledge of the system's structure.

Purpose of the Study:

• To develop an information-theoretic approach for generating reduced-order models of Markov chains.
• To enable model reduction without prior specification of the number of reduced states.
• To provide a data-driven method for optimizing model reduction parameters.

Main Methods:

• Utilizing two information-theoretic processes for model reduction.
• Employing a negative, modified Kullback-Leibler divergence to compare stationary chains.
• Solving a value-of-information criterion to probabilistically partition states.
• Introducing a single free parameter to control partition uncertainty and group number.

Main Results:

• A novel method for obtaining reduced-order Markov chain models.
• Probabilistic partitioning of states based on information-theoretic criteria.
• A data-driven approach to select the optimal free parameter for model reduction.
• Successful simplification of high-order Markov chains without pre-defining the number of states.

Conclusions:

• The proposed information-theoretic approach offers an effective way to reduce the complexity of Markov chains.
• The data-driven parameter selection method enhances the practicality and applicability of the technique.
• This work provides a valuable tool for analyzing and simplifying complex dynamical systems modeled by Markov chains.