Monday, October 21, 2013
Key Ballroom Foyer (Hilton Baltimore)
Poster Board # P2-42
Quantitative Methods and Theoretical Developments (MET)
Candidate for the Lee B. Lusted Student Prize Competition
Hawre Jalal, MD, MSc1, Jeremy D. Goldhaber-Fiebert, PhD2 and Karen M. Kuntz, ScD1, (1)University of Minnesota, Minneapolis, MN, (2)Stanford University, Stanford, CA
Purpose: Regression metamodeling (RM) is
a useful technique for efficiently revealing parameter sensitivities in a model
in terms of marginal effects of each parameter on policy-relevant outcomes
using the output from probabilistic sensitivity analysis (PSA). The present
study examined the performance of RM when model parameters are correlated.
Methods: A metamodel is a statistical model
that can summarize model parameter sensitivities from PSA by regressing model
outcomes on the model input parameters. Coefficients from RM can be
interpreted as changes in the model outcome due to a change in the
corresponding input. Decision models with two or more parameters that are
highly correlated may present an important limitation of RM, as collinear
variables do in multivariate regression. Increased correlation in RM parameters
may widen the confidence intervals of the coefficient estimates. We used a
previously published model of treating herpes simplex encephalopathy, where the
outcome is the expected utility of undergoing a brain biopsy. We incorporated a
correlation between two of the model parameters: the probability of dying
because of biopsy (pDieBiopsy) and the probability of developing severe complications
following biopsy (pSevereBiopsy). We ran 10,000 PSA simulations for each hypothetical
correlation level between these two parameters, varying the correlation value (rho)
from 0 to 1. We then examined the precision of the estimated RM coefficients
at each value of rho.
Results: The figure shows the RM
coefficients of pDieBiopsy and pSevereBiopsy (solid line), and their confidence
intervals (gray region) for various correlation coefficient values. The
negative coefficients indicate that an increase in the value of either parameter
results in a reduction in the expected utility of biopsy. The confidence intervals
maintains the same width at various correlation levels except for near exact correlation
(i.e., when rho = 1). We found similar results with other correlated
parameters.
Conclusion: We found RM to accurately
predict parameter sensitivities except when these parameters are in near
perfect correlation. In these situations, including correlated parameters in
the model may be unnecessary because one of the correlated parameters can be
expressed as a function of the other one. Using standard regression diagnostics
(e.g., the condition index) to identify situations of high multicollinearity
may be appropriate when performing RM.
