Purpose: The purpose of this study is to review existing sensitivity analysis methodologies for decision models, organize the methods into a logical typology, critically analyze advantages and disadvantages of existing methods, present two novel approaches that address limitations, and illustrate and discuss the two novel approaches in the context of a generic medical decision problem.
Methods: We conducted a systematic review of existing approaches to sensitivity analysis for decision models found in the medical decision making and the operations research literature. We identified ten generic methods along with associated graphical presentation methods. The methods can be grouped into three general approaches: 1) a prior free, or deterministic, approach based on parametric analysis; 2) a single prior approach based on probabilistic analysis; and 3) a Bayesian approach which generalizes the latter approach.
Results: We point out limitations found in current sensitivity techniques. In particular, all existing methods present flaws in determining and understanding "close calls" in decision problems. We propose two novel alternatives. The first technique builds upon regret theory and the concept of maximum relative expected loss. It generalizes the flat maxima approach that has been used in the past for decision trees. The second technique directly accounts for differences in expected utilities between competing decision strategies. In this approach, one first calculates the probability that the “optimal” strategy wins by at least a given amount over the next best strategy. Then, one varies the winning amount to construct a decumulative distribution of the expected winning amount. Finally, the distribution can be analyzed to explore sources of sensitivity and can be summarized into specific measures of sensitivity. It also lends itself to a simple and elegant graphical representation. Both techniques are easily implementable within current software used to develop and analyze complex decision problems. Both approaches are illustrated with a specific medical decision problem.
Conclusions: In this methods paper, current sensitivity analysis techniques are reviewed and are categorized in three complementary generic approaches. All present advantages and disadvantages that can be addressed with two novel techniques. In particular, one approach successfully addresses all issues and provides a novel way to fully describe, quantify, analyze, and understand the sensitivity of any complex decision problem.
See more of: The 34th Annual Meeting of the Society for Medical Decision Making