EXCLUSION CRITERIA IN NATIONAL HEALTH STATE VALUATION STUDIES: A SYSTEMATIC REVIEW

Purpose: To provide a systematic examination of exclusion criteria used in the derivation of societal scoring algorithms using data from national valuation studies.

Methods: A comprehensive literature search was undertaken, including MEDLINE, instrument websites and publication reference lists. Studies that used data from national valuation exercises (standard gamble, time trade-off or visual analogue scale techniques) and report a scoring algorithm for a generic preference-based health-related quality of life (HRQoL) measure were included. Data extraction focused on the adopted exclusion criteria, justification for the adopted criteria, and the number and sociodemographic characteristics of excluded responses/respondents.

Results: One thousand and fifty-seven unique records were identified. In total, 70 papers met the inclusion criteria; 54 (77%) studies were primary analyses of national valuation exercises, with the remaining 16 (23%) reporting secondary analyses of data. The 70 studies comprised analyses for the EQ-5D (3-level and 5-level), SF-6D (SF-12 and SF-36), Health Utilities Index (HUI2 and HUI3), 15D, Quality of Well-Being Self-Administered (QWB-SA) and Assessment of Quality of Life (AQoL) instruments (4-, 6-, 7- and 8-dimension variants). In addition to logical inconsistencies, respondents were often excluded if they valued fewer than three health states or if they gave the same value to all health states. Numerous other exclusion criteria were identified, with varying degrees of justification. Some criteria were based on the assumption that respondents did not understand the task, whereas others were associated with the chosen statistical modelling techniques. The number of excluded respondents ranged from 0% to 65%. Excluded respondents tended to be older, less educated, and less healthy.

Conclusions: There is considerable variation in the exclusion criteria used by analysts when deriving societal scoring algorithms for preference-based HRQoL instruments. The trade-off inherent in this topic is between data quality and the representation of societal preferences. On one hand, inconsistent responses are excluded from datasets on data quality grounds – a defensible position, depending on the definition of inconsistencies – whereas, conversely, such exclusions decrease the true representativeness of societal preferences if certain sociodemographic groups are systematically excluded. The findings of this review suggest that further consideration is needed regarding exclusion criteria for health state valuation exercises, and whether standardized criteria should be developed.