SENSITIVITY OF TREATMENT DECISIONS TO BIAS ADJUSTMENT IN NETWORK META-ANALYSIS

Purpose:

   We demonstrate a new method for quantifying the effects of bias adjustment on treatment decisions based on a network meta-analysis (NMA).

Method(s):

   NMA combines evidence on multiple treatments from several studies to provide internally consistent treatment effect estimates and is frequently used to inform clinical guideline recommendations. Evidence from included studies is typically assessed for risk of bias using subjective tools and checklists; however these provide no information on the effects of potential bias on decisions based on the results of the NMA. 

   We propose a new method that provides quantitative assessment of the effects of potential bias adjustments, by deriving bias-adjustment thresholds which describe the smallest changes to the data that would result in a change of treatment decision. In other words, the treatment decision is invariant to biases within the threshold limits. Bias adjustments can be considered for individual study estimates or for overall treatment contrasts. 

   Bias-adjustment thresholds are derived by manipulating the Bayesian joint posterior resulting from the NMA. The amount that a given data point can change before affecting the treatment decision depends upon the influence of that data point on the joint posterior.

   We also assess the effects of bias adjustment in a probabilistic cost-effectiveness analysis using inputs from the NMA. We then assess the sensitivity to bias of a treatment decision based on net benefit.

Result(s):

   The threshold method was applied to a series of examples from published NICE guidelines. In most cases the treatment recommendation was robust to plausible levels of bias in all but a small proportion of contrasts or studies. In larger, well connected networks with large numbers of trials, recommendations were robust against almost any plausible bias adjustments. Sensitivity to bias adjustments for net benefit decisions resulting from cost-effectiveness analysis was also considered, showing similar results.

Conclusion(s):

   Threshold analysis provides insight into the effects of bias adjustment on treatment decisions. Applying the method to treatment contrasts confers considerable flexibility, since practical applications are often based on complex models with multiple types of data input. We can have more confidence in treatment recommendations where bias-adjustment thresholds are large, and focus attention on the quality of decision-sensitive trials and contrasts, potentially reducing the need for laborious critical appraisal of all included trials.