MAKING DECISIONS UNDER UNCERTAINTY: THE VALUE OF ECONOMIC EVALUATIONS IN THE ASSESSMENT OF NON-DRUG HEALTH TECHNOLOGIES IN CANADA

Purpose: The value of decision analytic models extends beyond generating a single point estimate to  systematic examination and reporting of uncertainty and its impact on relevant outcomes.  When conducting economic evaluations to support decision making, CADTH explores the impact of uncertainty within de novo models using a number of different methods. The aim of this study is to understand how uncertainty has been handled and presented in the assessments of non-drug technologies at CADTH and how this has been used to facilitate decision making.

Methods: Most recent economic evaluations conducted by CADTH for non-drug technologies were selected for this review: end-tidal CO₂ monitoring; deficient mismatch repair screening in patients with colorectal cancer; and, non-surgical treatments for obstructive sleep apnea. The types of uncertainty in each evaluation were identified and categorized according to the ISPOR-SMDM Modeling Good Research Practices Task Force Paper; methods used to address uncertainty were reviewed; and,  potential implications of providing this information to inform decision making were considered.

Results: The selected cases highlighted some specific challenges of modelling non-drug technologies including: lack of comparative clinical data; paucity of information (e.g., natural history of disease, costs); spillover effects that impact other health care use; incremental innovation; and broader clinical applications of technologies in the presence of treatment or disease heterogeneity. Parameter and structural uncertainty were common to all models. All economic evaluations reported the results probabilistically. While deterministic sensitivity analyses were presented to address parameter uncertainty (end-tidal CO₂ monitoring), given the potential of misinterpreting these results in decision making, an alternative method of one-way stochastic sensitivity analyses have been performed in more recent evaluations. Structural uncertainty was addressed through scenario analyses, subject to the availability of data. CADTH’s advisory body for non-drug technologies (HTERP) often included discussions on uncertainty when formulating recommendations.

Conclusions: Given the challenges in the assessment of non-drug technologies, the assessment of uncertainty plays an important role in decision making at CADTH and understanding where evidence gaps exist.  Some challenges remain when accounting for uncertainty, especially in the context of little to no evidence.