DISTRIBUTION OF THRESHOLDS IN REPEATED MONTE CARLO SIMULATIONS

Purpose: Cost-effectiveness acceptability curves (CEAC) demonstrate the likelihood that an alternative is cost-effective at different values of an outcome, accounting for stochastic uncertainty in model parameters.  However, the value of the outcome is not the only parameter for which a threshold for policy action may need to be determined.  To date, most threshold sensitivity analyses focusing on parameters other than WTP for the outcome have been conducted at point estimates for other parameters.  We show how policy-relevant thresholds can be calculated using a probabilistic framework similar to CEACs, and how this might inform the hypothesis for a study to detect differences from the threshold.
Methods: We built a decision tree to describe two treatments for a hypothetical condition: (1) immediate surgery; (2) more complex surgery after allowing time for spontaneous resolution. First, a threshold analysis was conducted using parameter point estimates to determine the probability of spontaneous resolution at which the alternatives would have equal costs. Second, probabilistic uncertainty around the threshold  was determined using Monte Carlo simulation. The relationship between the uncertainty of the threshold and the outcome value was explored.
Results: The point estimate of the probability of spontaneous resolution above which a delayed surgery approach would have lower costs was 76%.  In 63% of 1000 repeated simulations, delayed surgery was less costly when the spontaneous resolution rate was 76%.  Delayed surgery was never less costly if the probability of spontaneous resolution was 71% or less and was always less costly if the probability of spontaneous resolution was 79% or more.  When the outcome was assigned a positive value a comparison of net benefits resulted in which the gap between the values at which delayed surgery was never economically preferable and the values at which it was always preferred grew wider.
Conclusions: This analysis demonstrates the uncertainty of a threshold.  A study designed to detect whether a spontaneous resolution rate is greater than the point estimate of the threshold would detect whether the spontaneous resolution rate was greater than the rate making delayed surgery less costly only 63% of the time.   Necessary certainty depends on decision makers’ risk aversion.  Analysts conducting threshold analyses accounting for stochastic uncertainty must account for decision makers’ need for certainty and value of the outcome.