Identifying the generator matrix of a stationary Markov chain using partially observable data

Area of Science:

• Mathematics
• Probability Theory
• Computational Statistics

Background:

• Real-world systems often have inaccessible states, complicating analysis of stationary Markov chains.
• Identifying the generator matrix is essential for understanding system dynamics but challenging with partial observations.

Purpose of the Study:

• To develop methods for identifying the generator matrix of stationary Markov chains using minimal observations.
• To generalize the concept of hitting-time distributions for irreversibly stationary Markov chains.

Main Methods:

• Generalization of hitting-time distributions from reversible to irreversibly stationary Markov chains.
• Decoding the generator matrix using taboo rates and observations from specific states (leaves and adjacent states).
• Development and validation of algorithms for accurate generator matrix calculation.

Main Results:

• The generator matrix of reversible and specific irreversibly stationary Markov chains can be identified under mild conditions using limited observations.
• Proposed algorithms demonstrate accuracy and efficiency in numerical examples.
• The study confirms that partially observable data is sufficient for identifying the generator matrix.

Conclusions:

• This research provides a novel approach to solving the inverse problem for stationary Markov chains with inaccessible states.
• The findings have practical implications, as demonstrated by an application in neurophysiology.
• The methods enable the reconstruction of system dynamics from incomplete observational data.