Purpose: There is rising interest in estimating health-state valuations from ordinal data, although key methodological challenges remain. Recent studies suggest that combining information elicited using ordinal and cardinal techniques may optimize tradeoffs between simpler data collection and valid estimation. Analytic models for ordinal data typically assume constant variance across states and unbiased measurement. This study used simulation and analysis of a large empirical data set to quantify the potential importance of three departures from these standard assumptions.
Method: We applied a probit model that combines information from paired comparisons of heath states with discrete choice time trade-off (TTO) responses. The model assumes that values for health states on a (0,1) scale are compressed at either end of the scale, such that the log-odds of these values are normally distributed with constant variance. We generated simulated datasets that deviated from these assumptions and then tested the robustness of the analytic models to these deviations. Specifically, we simulated (1) heteroskedastic variance (in log-odds space) across health states; (2) bias due to non-detectable differences between similar health states; and (3) strict preference for longevity over perfect health in TTO responses among some proportion of respondents (“never traders”). In light of the results of the simulation study we tested alternative modeling strategies in a large, multi-country empirical database on health valuations.
Result: For simulations in which all assumptions were met (and given a typical sample size for health valuation) the model accurately estimated health-state values, with a mean relative error (MRE) of 6% for predicted vs. true values. If variances for intermediate states were higher than for extreme states (by a factor of 4 in log-odds space), the model still performed well (MRE=10%). Results were more sensitive to the other two departures. For example, when 15% of paired comparisons produced random errors due to non-detectable differences, MRE rose to 25%, and when 20% of respondents were “never traders”, MRE was 23%. Addressing the problems of non-detectable differences and “never traders” in empirical data demanded both larger numbers of TTO responses and augmented analytic models.
Conclusion: Ordinal approaches to health-state valuation are attractive alternatives to existing metric methods, but further work is needed in identifying the most appropriate methods for data collection and analysis.
Candidate for the Lee B. Lusted Student Prize Competition