DOES INTRODUCING IMPRECISION AROUND PROBABILITIES FOR BENEFIT AND HARM INFLUENCE THE WAY PEOPLE VALUE TREATMENTS?

Purpose: In contexts where sampling error varies between treatment alternatives (e.g. similar average risks, but one treatment has a larger confidence interval), patients can fail to choose what option is best for them unless this imprecision is conveyed and understood. The aim of this study is to describe how two methods for conveying imprecision in risk influence peoples treatment decisions using a discrete choice experiment (DCE).

Methods: We randomized a representative sample of the Canadian general population to one of three surveys which sought choices between hypothetical treatments for rheumatoid arthritis based on different levels of 7 attributes: route and frequency of administration, chance of benefit, serious and minor side-effects and life expectancy, and imprecision in benefit and side-effect estimates. The surveys differed in the way imprecision was described: 1) none, 2) quantitative description based on a range with a visual prop, 3) qualitative description simply describing the confidence in the evidence using wording from the GRADE group. Each respondent answered 10 pairwise DCE questions by indicating their strength of preference for each treatment, where each treatment had differing levels for each attribute as generated by a D-efficient experimental design. Given the differing complexities between surveys, we used a heteroscedatic conditional logit model to enable the variance of unobserved factors to vary between surveys. We estimated marginal willingness to give up life for each of the attribute levels in each survey.

Results: Of the 2011 respondents that provided rational responses to the survey the mean age was 45, 57% were female, and 44% had up to a high school education. Conditional logit models suggested people placed value on imprecision when conveyed qualitatively (MRS=2.18 for high vs low confidence, p<0.001) but not quantitatively (MRS= 0.17 for high vs low confidence, p=0.675). Both qualitative and quantitative methods led to small but significant increases in decision uncertainty for choosing any treatment (61% vs 66% having strong preferences for options, p<0.001).

Conclusions: Imprecision in estimates of benefits and harms around treatment choices is rarely described to patients. These findings suggest qualitative descriptions of imprecision may provide people with the gist of the evidence for treatment effects, though since this was a descriptive analysis, we do not know whether these choices were truly informed.