PROBABILISTIC SENSITIVITY ANALYSIS IN SEQUENTIAL DECISION MODELS

Purpose: Sequential decision problems with embedded decisions are frequently encountered in medical decision-making, and are commonly solved using Markov decision processes (MDPs). ISPOR-SMDM Modeling Good Research Practices Task Force recommends conducting probabilistic sensitivity analysis (PSA) in decision-analytic models; however conducting PSA in models with sequential decisions is challenging and has not been explored. We propose an approach of conducting PSA to assess confidence in the results obtained from such sequential decision models.

Method: We illustrated our approach using a model where three HIV drugs (A, B, C) were allowed to switch at 10 sequential decisions points (ie, 9 embedded decisions). We solved the model as a discrete-time finite-horizon MDP to find the most cost-effective treatment sequence (ie, the optimal policy) out of 206 trillion feasible sequences. We defined uncertainty in our model parameters using the recommended statistical distributions. We first evaluated sensitivity of each parameter by sampling from one parameter at a time and keeping all others fixed (i.e., a univariate approach). For each parameter, we solved our MDP for (optimal) policies 10,000 times using sampled parameter values, and then estimated its sensitivity by counting the percentage of sampled policies that were exactly the same as the baseline policy. Next, we evaluated the confidence in the net benefit of the optimal policy by simultaneously sampling from all parameters (i.e., a multivariate approach) and solving the MDP 10,000 times. We estimated the proportion of the samples where the baseline policy had a lower net benefit than sampled policies within an optimality gap, where the gap was defined as stakeholders' willingness to accept the (suboptimal) baseline policy at certain optimality tolerance level.

Result: Figure 1 presents the sensitivity of parameters, which are ordered from the most to the least sensitive. Figure 2 presents the probability of accepting the baseline policy for a given tolerance of optimality, which we call the policy acceptability curve. For instance, when stakeholders are indifferent to policies with objective values 20% less than the optimal value, the baseline policy is acceptable at 85% confidence.

Conclusion: We proposed an approach for assessing confidence in the optimal policy of sequential decision models, which to our knowledge has not been explored. Providing confidence in such models can increase their credibility among policymakers and other stakeholders.