Adding noise to Markov cohort state-transition model in decision modeling and cost-effectiveness analysis

Following its introduction over 30 years ago, the Markov cohort state-transition model has been used extensively to model population trajectories over time in health decision modeling and cost-effectiveness analysis studies. We recently showed that a cohort model represents the average of a continuous-time stochastic process on a multidimensional integer lattice governed by a master equation, which represents the time-evolution of the probability function of an integer-valued random vector. By leveraging this theoretical connection, this study introduces an alternative modeling method using a stochastic differential equation (SDE) approach, which captures not only the mean behavior but also the variance of the population process. We show the derivation of an SDE model from first principles, describe an algorithm to construct an SDE and solve the SDE via simulation for use in practice, and demonstrate the two applications of an SDE in detail. The first example demonstrates that the population trajectories, and their mean and variance, from the SDE and other commonly used methods in decision modeling match. The second example shows that users can readily apply the SDE method in their existing works without the need for additional inputs beyond those required for constructing a conventional cohort model. In addition, the second example demonstrates that the SDE model is superior to a microsimulation model in terms of computational speed. In summary, an SDE model provides an alternative modeling framework which includes information on variance, can accommodate for time-varying parameters, and is computationally less expensive than a microsimulation for a typical cohort modeling problem.