Purpose: We assess the relative impact of different parameterisations of treatment effects on the resulting decision, on decision uncertainty and on the expected value of conducting further research in order to reduce decision uncertainty, and demonstrate the use of model averaging to incorporate structural uncertainty about the parameterisation.
Method: We use a Bayesian approach to model estimation, model selection, and model averaging in the context of Cost-Effectiveness and Expected Value of Perfect Information (EVPI) analyses for asthma treatments. We use aggregate level data from a connected network of four treatments compared in three pair-wise RCTs. We assess the relative impact of several different parameterisations of treatment effects on the resulting decision, on decision uncertainty, and on the EVPI. Structural uncertainty about which parameterisation to use is accounted for using model averaging and we develop a formula for calculating the EVPI in averaged models. Marginal posterior distributions are generated for each of the cost-effectiveness parameters using Markov Chain Monte Carlo simulation in WinBUGS, or Monte-Carlo simulation in Excel.
Result: The standard errors of incremental net benefit using structured models is reduced by up to 8 or 9-fold compared to the unstructured model, and the expected loss attaching to decision uncertainty by factors of several hundreds. Model averaging had little impact on the optimal decision but there was considerable impact on the EVPI.
Conclusion: Alternative structural assumptions can have an overwhelming effect on model uncertainty and Expected Value of Information. Structural uncertainty can be accounted for by model averaging, and EVPI can be calculated for averaged models.
Candidate for the Lee B. Lusted Student Prize Competition