LOOPY NO MORE: PROBABILISTIC SENSITIVITY ANALYSIS IN MICROSIMULATION MODELS

Purpose: Probabilistic sensitivity analysis (PSA) is recommended by ISPOR-SMDM Modeling Good Research Practices Task Force and is a necessary step for undertaking value of information analysis. However, conducting PSA can be computationally challenging in individual-level state-transition models (I-STMs) because of two levels of uncertainty—first- and second-order. Published guidelines suggest a careful evaluation of a balance between the inner and outer simulation loops. Our purpose was to evaluate the need for such a balance and find the optimal combination to conduct PSA in I-STMs.

Method: We created a simple 4-state I-STM of HIV treatment, which has been commonly used for didactic purpose in literature. Second-order, parameter uncertainty was defined using the recommended statistical distributions for 13 model parameters, which included state transition probabilities, utility weights and costs. We conducted PSA multiple times using different combinations of inner (ie, first-order) and outer loops (ie, second-order)—(A) 10 outer and 1,000 inner, (B) 100 outer and 100 inner, (C) 1,000 outer and 10 inner, (D) 10,000 outer and 1 inner. The total number of computational runs was equal (ie, 10,000) in all four combinations. Using independent initial random-seeds, we ran PSA 20 times for each combination and obtained 20 sets of results and plotted 20 cost-effectiveness acceptability curves (CEACs). Our rationale was to determine variability in outcomes resulting from joint first- and second-order uncertainty. We also estimated standard error (SE) (from 20 runs) in mean costs and quality-adjusted life years (QALYs).

Result: Figure 1 shows the CEACs of 20 sets of PSA runs for each combination (A–D) of inner and outer loops. For combination-A, the probability of cost-effectiveness at $50,000 willingness-to-pay was between 0.10–0.80, depending upon the random seed used; whereas, the corresponding range from combination-D was between 0.47–0.48. The SE in average costs and QALYs from combination-A were 30–120 folds higher than that from combination-D.

Conclusion: To run PSA in I-STMs, the best combination of inner-outer loops is obtained by getting rid of the inner loop and maximizing the runs in the outer loop (ie, single patient runs). Our approach is easy to implement in any modeling package, and is particularly useful for complex I-STMs where computational burden could be an issue.