1C-3 PROBABILISTIC SENSITIVITY ANALYSIS IN MICROSIMULATION MODELS: FINDING THE RIGHT BALANCE

Monday, October 24, 2016: 2:30 PM
Bayshore Ballroom Salon F, Lobby Level (Westin Bayshore Vancouver)

Jagpreet Chhatwal, PhD, MGH Institute for Technology Assessment and Harvard Medical School, Boston, MA, Kan Li, MS, The University of Texas Health Science Center at Houston, Houston, TX, Elisabeth A.L. Fenwick, PhD, ICON plc, Oxford, United Kingdom, Mark S. Roberts, MD, MPH, University of Pittsburgh School of Medicine, Pittsburgh, PA and Andrew Briggs, DPhil, University of Glasgow, Glasgow, United Kingdom

Purpose: Probabilistic sensitivity analysis (PSA) is a recommended approach and is a necessary step for undertaking value of information analysis. However, conducting PSA can be computationally challenging in individual-level state-transition models (I-STMs) because of two levels of uncertainty: first- and second-order. Published guidelines suggest a careful evaluation of a balance between the inner and outer simulation loops. Our purpose was to evaluate the need for such a balance and to find the optimal combination to conduct PSA in I-STMs.

Method: We used a previously published and validated I-STM that evaluated the cost-effectiveness of hepatitis C treatment. Second-order, parameter uncertainty was defined using the recommended statistical distributions. We conducted PSA multiple times using five different combinations of inner (i.e., first-order) and outer loops (i.e., second-order) labelled as A-E. The total number of computational runs was equal to 1 million in all combinations. Using independent initial random-seeds, we ran PSA 20 times for each combination and obtained 20 sets of results and plotted 20 cost-effectiveness acceptability curves (CEACs). Our rationale was to determine variability in outcomes resulting from joint first- and second-order uncertainty. We also estimated standard error (SE) (from 20 runs) in mean costs and quality-adjusted life years (QALYs).

Result: Figure 1 A-E shows the CEACs associated with each of 20 sets of PSA runs for each combination of inner and outer loops. As the number of outer loops increases the variation caused by the random seed falls such that there is overlap across the CEACs.  For the combination with 1 million outer loops (Fig E), the variability in the CEACs is completely resolved (i.e. the CEACs completely overlap), however, these CEACs are skewed downwards, compared to the other combinations. This is because when only 1 inner loop is used it is not possible to average across the individuals and hence the full impact of 1st order uncertainty is retained.

Conclusion: To run PSA in I-STMs, the right balance between inner-outer loops is needed. Using extreme combinations will result in inappropriate results which would impact on decisions regarding cost-effectiveness and the value of further research. Our empirical analysis also indicates that under conditions of constrained computational time, using more outer loops than inner loops should be preferred.

figure.png

See more of: ORAL ABSTRACTS: ECONOMIC EVALUATION METHODS

See more of: 38th Annual North American Meeting of the Society for Medical Decision Making