Purpose To systematically review methods to characterise structural uncertainty, identify those that are applicable to decision analytic modelling and demonstrate their applicability using case study models exhibiting different forms of structural uncertainty.

Methods A review was undertaken to classify types of structural uncertainty and identify approaches to characterise these uncertainties. Methods were then applied to 4 case study models demonstrating different form of structural uncertainty: screening for age related macular degeneration (ARMD), glycoproteins for unstable angina (GPA), clopidogrel for occlusive vascular events (CLOP) and screening for oral cancer (OC). The feasibility and value of using the methods for the alternative types of structural uncertainty was evaluated.

Results The review identified methods for characterising structural uncertainty; however only a subset were applicable to decision analytic models. These are 1) model averaging where alternative models, with different specifications, are built, and their results averaged, 2) computing results for each alternative model specification and to presenting these alternative results as scenario analyses and 3) parameterising the uncertainty directly in the model by including ‘uncertain' parameters. Running scenarios and model averaging was undertaken for all 4 case studies. Parameterising the structural uncertainty was only undertaken in those models were it was technically feasible; ARMD and OC.

In some cases structural uncertainty had little impact on cost-effectiveness e.g. in the ARMD model the ICER ranged from £12,892 to £16,176 for the 20/40 subgroup) and in some it had serious impact on cost effectiveness e.g. in the choice of the most cost effective strategy changed from strategy 1 was (ICER =£5,738) strategy 5 (ICER = £3,966). However, the value of additional research (EVPI) was particularly sensitive to structural uncertainty, in particular model averaging and parameterising uncertainty; because it introduced additional uncertainties; increased the EVPI quite substantially.

Neither running scenarios nor model averaging helped to inform a decision about the requirement for more evidence to reduce any structural uncertainties. Parameterising the uncertainty directly in the model, can inform this decision, but unfortunately cannot be undertaken for more complex structural uncertainties such as adding additional comparators.

Conclusions It is clear that alternative structural assumptions produce very different values of additional research estimates and it is therefore essential, for decision making purposes, to incorporate issues of structural uncertainty into the decision modelling process.