MEASURING DECISION SENSITIVITY WITH MULTINOMIAL LOGISTIC REGRESSION METAMODELING

   Purpose:   Modelers lack a simple tool to examine decision sensitivity (i.e., the change in the probability of a strategy being optimal due to parameter uncertainty).  We propose multinomial logistic regression (MNR) metamodeling to reveal decision sensitivity.       Methods:  MNR is useful in analyses where the dependent varaible is categorical and not ordered. In this study, we apply MNR in a novel way to analyze the probabilistic sensitivity analysis (PSA) from a decision model in order to reveal decision sensitivity.  We demonstrate our approach with a previously published decision model for treating a suspected case of herpes simplex encephalopathy.  The model compares three strategies: treat everyone, biopsy, and do not treat or biopsy.  We performed 10,000 PSA scenarios.  For the MNR, we treated the model's input parameter values as independent variables and the optimal strategy in each iteration as the dependent variable. In this capacity the MNR is a second (meta) model.  Because the regression coefficients are difficult to interpret, we report the marginal effects (ME) as a direct measure of decision sensitivity.  The MEs measure the change in the probability of each strategy being optimal due to one unit change in each parameter.  Furthermore, we developed a new score, the sum of absolute marginal effects (SAME) to combine the ME of a parameter on all the strategies, and compared our results to expected value of partial perfect information (EVPPI).       Results:   The probability of severe sequalae following biopsy was associated with the highest decision sensitivity.  The ME of this parameter on biopsy was -0.28, indicating that the probability of biopsy being optimal decreases by 0.28 if the value of this parameter is increased by one standard deviation from its mean.  Similarly, the importance of all the model parameters were ranked by their ME and SAME scores.  In addition, the SAME scores were highly correlated with the EVPPI (correlation coefficient = 0.97) (see Figure).       Conclusion:   Regression analsyis can be used to evaluate the impact of decision model parameters and is highly correlated with EVPPI results.