Purpose To demonstrate how formal elicitation methods can be used to parameterise structural uncertainties within decision analytic models and evaluate alternative methods to synthesise the information acquired through elicitation.

Background Current methods to characterise structural uncertainty (scenario analysis and model averaging) fail to provide quantitative estimates of the scale of the uncertainty or inform the question of what further evidence would be needed to resolve them. A parameterisation of the structural uncertainty, and elicitation and synthesis of formal priors on these new model parameters is required. It is then possible to conduct expected value of information analysis and establish the value of acquiring further evidence to resolve structural uncertainty.

Methods Structural uncertainties were identified in an existing probabilistic model of etanercept and infliximab for psoriatic arthritis. An interactive elicitation exercise was designed to: generate estimates of disease progression while responding to treatment and while relapsing; examine the extent of correlation between these two parameters and response to treatment; and calibrate expert judgement in subsequent synthesis. Fifteen experts completed the questionnaire. Alternative methods of synthesise were evaluated: scenarios using individual experts; random sampling across experts (linear pooling); and meta-analysis using fixed or random effects models. The resulting estimates of the structural parameters were applied to the model and the value of information associated with the structural uncertainties was calculated.

Results Responses to the elicitation questions varied, reflecting different clinical opinion regarding treatment. Responses to the known parameter showed that experts assessments could be regarded as differentially weighted, with some experts providing more a more accurate response than others. When applied to the model, using expert's assessment as individual scenarios produced 15 different sets of results and estimates of EVI which is of little value to decision makers. Sampling across experts generated the highest EVI (£12.6mil) compared to the random effects model method (£5.5mil) and fixed effects model method (£4mil). Tests for heterogeneity showed that a random effects model was more appropriate for pooling than a fixed effects model (chi-squared = 31.07).

Conclusions Parameterising structural uncertainty requires formal methods of elicitation which can capture correlation and assess the quality of responses. The appropriate synthesis of expert judgement is important so that the value of collecting further data to resolve structural uncertainties can be estimated.