INCORPORATING QUALITY OF EFFICACY DOCUMENTATION IN HEALTH ECONOMIC EVALUATIONS – DOES IT MATTER?

Background:     When conducting health economic evaluations, efficacy documentation is usually based on either a randomized controlled trial (RCT) or several RCTs combined in a meta-analysis. Usually meta-analyses increase accuracy regarding outcomes. A single RCT however, may in some cases have narrower confidence interval than a meta-analysis due to heterogeneity between trials. This contradiction is one of several which is brought to light by grading the quality of the documentation. Probably the most frequently used grading tool today, is GRADE (Grading of Recommendations Assessment, Development and Evaluation). This is the tool used in Cochrane reviews and also by some health technology assessment agencies.    A Bayesian framework is often applied to economic evaluations. In a Bayesian framework, models for economic evaluation should be based on parameters with uncertainty described by a probability distribution, rather than fixed parameters. For efficacy parameters based on meta-analyses, it is common to use probability distributions based on the mean and standard error from a meta-analysis. This does however, not reflect the uncertainty related to study quality.

Purpose:    To explore the value of incorporating study quality in health economic evaluations.

Method:    We incorporated quality of the documentation (assessed through GRADE) into efficacy parameters in models developed by the Norwegian Knowledge Centre for the Health Services (NorCaD, MOON, COSMO and MOCCA). For efficacy estimates based on high quality documentation, probability distributions were based on 95% confidence intervals. For moderate, low or very low quality, we assumed confidence intervals in reality reflected 90%, 80% and 70% confidence intervals. Models were run both with GRADE incorporated and without to show whether there was any difference in cost-effectiveness.

Result:    We performed cost-effectiveness analyses which showed substantial differences in results when GRADE was incorporated, compared to when GRADE was not incorporated. In some analyses, conclusions were completely changed. We also performed some value of information analyses which showed that incorporating quality of outcomes in analyses would imply that it is more cost-effective to conduct new research.

Conclusion:    Incorporating GRADE into health economic evaluations significantly change results and changes the uncertainty around the conclusions. If GRADE is an appropriate tool to assess the quality of outcomes, then we believe that inclusion of GRADE in health economic evaluations is more appropriate than not including it.