Stationary and non-stationary transition probabilities in decision making: Modeling COVID-19 dynamics

This study present a comparative modeling framework for COVID-19 dynamics using stationary and non-stationary transition probabilities within a Markov decision process (MDP). Stationary transitions assume constant rates, while non-stationary transitions capture time-dependent behaviors driven by policy interventions or behavioral changes. We develop a seven-compartmental epidemiological model, derive transition probabilities from binomial and multinomial processes, and implement time-dependent parameterizations to reflect real-world dynamics. Mathematical models for both stationary and non-stationary transition frameworks are developed and simulated over a 365-day period to emphasize dynamic variations in epidemic outcomes. Our findings highlight the significance of non-stationary modeling in accurately representing the dynamic characteristics of pandemic situations and provide recommendations for optimizing public health interventions under uncertainty. This comparative analysis offers useful information for epidemiological modeling and decision making in dynamic risk environments.