ANALYTICAL SOLUTION METHODS FOR CONTINUOUS-TIME MARKOV AND SEMI-MARKOV MODELS

Purpose: To show how continuous-time Markov and semi-Markov models can be analyzed without simulation, based on matrix algebra and stochastic process methods.

Method: Markov and semi-Markov decision models are widely used for cost-effectiveness analysis in health-economic evaluation. These models are often evaluated in discrete time using cohort analysis or in continuous time using microsimulation. However, both approaches have limitations. Cohort analysis is based on the assumption that at most 1 event can occur per cycle and requires ad-hoc methods to avoid biased cost-effectiveness estimates. Microsimulation introduces simulation error and can be computationally intensive, especially when used for model calibration and probabilistic sensitivity analysis. We use matrix algebra and stochastic process methods to derive analytical solutions for continuous-time Markov models. We also show how semi-Markov models can be approximated by Markov models, so that semi-Markov models can also be analyzed without microsimulation.

Result: Using Kolmogorov’s differential equations, we find analytical solutions for the expected distribution of patients over the health states in Markov chain models, and the expected time spent in each state. These mathematical results enable us to analytically calculate the expected costs and health effects of continuous-time Markov chain models. This method can be interpreted as a continuous-time version of the fundamental matrix solution. This method can also be used to account for age-specific transition rates and discounting, which was not possible using the original fundamental matrix solution. Finally, we show how the concept of tunnel states can be generalized so that semi-Markov models (i.e., with any type of sojourn time distribution) can be approximated by Markov models with any degree of accuracy. Computational tests confirm that this approach is feasible; it is possible to compute the costs and health effects in continuous-time Markov models with hundreds of states within a few seconds.

Conclusion: Continuous-time Markov and semi-Markov models are a versatile tool for estimating the health and economic effects of medical interventions. Currently, these models are almost always evaluated using microsimulation. However, analytical solution methods exist and can easily be implemented. Analytical solutions can simplify the optimization methods used for model calibration and can reduce the computation time needed for probabilistic sensitivity analyses.