16th Biennial European Conference

June 12 - 14, 2016

Purpose: Evidence-based medicine and shared decision making are central tenets of modern health care. Sharing information with patients is presumed to improve patients’ understanding of their options and to increase health care providers’ understanding of patient preferences.  Health care providers may share different types of evidence with their patients, including statistical evidence based on scientific studies and narrative evidence based on their experiences.  Studies have shown that especially low-numerate patients are swayed by anti-vaccine narratives from other patients.  However, patients’ responses to pro-vaccine or physician-provided narratives have rarely been evaluated.  Here, we examined how recipients’ numeracy was related to their responses to pro-vaccine and anti-vaccine narratives from different sources, as well as to the process through which the presented narratives influenced their vaccination intentions.

Method(s): We recruited 1113 participants from a US national internet panel. They received different types of narratives about flu shots, in addition to a standard pamphlet from the Centers for Disease Control and Prevention (CDC).  Participants were randomly assigned to a narrative that was (a) pro-vaccine or anti-vaccine, (b) presented by either a patient discussing their own experience, a physician discussing another patient’s experience, or a physician discussing the experience of 50 patients, and (c) presented before or after the CDC pamphlet.  Pro-vaccine narratives described the flu experiences of patients who got the flu after not getting a flu shot, and anti-vaccine narratives described the flu experiences of patients who got the flu after getting a flu shot.  Narratives were equivalent in length and wording.  Participants subsequently indicated the probability that they would get vaccinated, perceptions of flu risk and severity with and without the flu shot, and their ratings of the quality of the presented narratives.

Result(s): Low-numerate individuals reported lower vaccination intentions, especially after reading anti-vaccine (vs. pro-vaccine) narratives.  A multi-mediation analysis suggested that low-numerate individuals’ vaccination intentions were reduced by anti-vaccine narratives and boosted by pro-vaccine narratives, due to their perceiving narratives as relatively better than did high-numerate individuals.  These findings held across other conditions, including whether narratives were provided by patients or physicians.

Conclusion(s): Health-care providers may add narrative information when presenting statistical evidence to inform patients’ decisions.  As compared to high-numerate recipients, low-numerate recipients seem to rely more on such narrative information when making their decisions.

Purpose: Price savings can be a salient factor for consumers as they consider drug options; advertisers sometimes include price-comparison information in direct-to-consumer (DTC) prescription drug ads. Such comparisons may inadvertently imply superiority or equivalence of a drug’s efficacy or safety, affecting potential for informed decision-making. A context statement – a disclosure noting compared products may or may not be equally effective or safe or different in afforded savings – is intended to correct such inferences. We investigated whether a context statement is useful in this regard.

Method(s): Using an experiment with 1,490 consumers self-identified with diabetes, we compared participants who saw a context statement in a fictitious (but professionally developed) price comparison DTC drug ad with participants who did not. We also included a control group that saw a DTC ad without the price-comparison or context statements.

Result(s): A majority of participants (59% of those assigned to see it) did not accurately report seeing the context statement. This pattern emerged despite the context statement being presented prominently below the price claim in the top half of the ad, linked with an asterisk.

We also assessed perceptions of participants who did recognize the presence of the context statement. Among confirmed exposure participants, a one-way ANOVA revealed a main effect of condition on perceptions of whether Veridan (the fictitious drug advertised) and Lyrica (the actual drug to which Veridan was compared) are interchangeable, F(2, 899) = 9.57, p<.001. Planned contrasts revealed that consumers seeing the context statement were more likely to agree the ad communicated uncertainty regarding comparative risk and efficacy than consumers seeing the comparison without context statement (M =4.48 vs. 4.07, p<.002). 

The context statement also noted price savings presented may not reflect actual savings by consumers or third-party payers. Among confirmed exposure participants, those seeing the context statement rated the price comparison as less accurate (M = 4.44, SD = 1.20) than those seeing the comparison without the context statement (M = 4.79, SD = 1.19), F(1, 472) = 9.351, p<.01.

Conclusion(s): When people read a context statement, they demonstrate intended uncertainty about risks, efficacy, and savings. Despite its prominence and placement, however, the majority did not notice the statement. Although results support the potential for developing comprehensible context statements to clarify price comparisons, consumer attention may limit their effectiveness.

Purpose: Recent evidence showed that ‘1-in-X’ ratios (e.g., 1 in 12) triggered higher subjective probability than ‘N-in-X*N’ ratios (e.g., 3 in 36) when interpreting health-related risks on a verbal probability scale (hereafter, ‘1-in-X’ effect), however it is not clear whether the ‘1-in-X’ formats overestimate or the ‘N-in-X*N’ formats underestimate the objective probabilities. Therefore, we aimed to establish which format lead to more accurate probability perception.

Method(s): In five parallel-designed experiments, 975 participants from a general adult population (54.7% women, M = 35.0, SD=11.2 years old) were randomly allocated to one of five experiments. Each experiment followed a 2(format) × 4(scenario) mixed design. In each experiment, participants read the risk of contracting a disease during their trip abroad. The risk was presented either in a ‘1-in-X’ or ‘N-in-X*N’ format in four scenarios randomly presented: malaria, Ebola, flu, Lyme disease. Participants assessed the risk on either a verbal probability scale (Exp. 1), a numerical probability scale (0-100%, Exp. 2), an arbitrary frequency scale (X out of 286, Exp. 3), a numerical probability scale with a delayed presentation (Exp. 4) or an arbitrary frequency scale with a delayed presentation (Exp. 5). Participants also made decision whether to cancel the trip to respective countries.

Result(s): We replicated the ‘1-in-X’ effect when probability perception was measured with a verbal probability scale (Exp. 1, Hedges’ g = 0.63, 95% CI[0.35, 0.91]). In the remaining numerical scale experiments (Exp. 2-5), we found that both ratio formats led to probability over-estimation (on average by 5.2%, 95% CI[2.1%, 8.4%], estimated in a multilevel meta-analysis). The ‘1-in-X’ formats triggered consistently higher subjective probability than ‘N-in-X*N’ formats: multi-level meta-analytical effect was g = 0.18, 95% CI[0.05, 0.32]. The ‘1-in-X’ ratio formats affected participants’ decision-making as they led to a higher willingness to cancel the trip abroad, aggregated effect across scenarios in Exp. 1-5, g= 0.17, 95% CI[0.04, 0.29].

Conclusion(s): Participants overestimated actual objective probabilities in both ratio formats. Since the ‘1-in-X’ effect was observed in all numerical scales (Exp. 2-5), bigger overestimation occurred in the ‘1-in-X’ format conditions. Health professionals should use ‘1-in-X’ formats with caution, because they make medical probabilities look bigger than they really are and, in turn, affect related decision-making.

Purpose: Medical professionals and the public are poor at solving Bayesian inference problems when presented in the form of conditional probabilities (e.g., determining the probability that an individual has a disease given that they tested positive).  When the same information is presented as natural frequencies, performance is improved.  The current meta-analysis sought to reconcile twenty years of research on natural frequencies, to clarify what is a natural frequency and to identify when, why, and for whom the format is most effective. 

Method(s): To identify papers that compared conditional probability and natural frequency Bayesian reasoning tasks, we conducted a systematic review across three major scientific and medical databases: Ovid(psych), Web of Science, and Pubmed.  Cited reference searches were conducted on key papers and we requested relevant or unpublished papers from the JDM mailing list that we may have otherwise missed in our systematic search.  Thirty relevant papers were identified from which 90 effects were subsequently analysed.

Result(s): A broad range of potential moderators were coded.  These included moderators related to individual characteristics (e.g., numeracy, education), problem representation (e.g., congruence between answer and problem format, use of visual aids, menu), and methodology (e.g., use of incentives, scoring protocol).  Results revealed the expected natural frequency facilitation effect: on average, performance was enhanced for natural frequency formats when compared to conditional probabilities.  A number of variables moderated the effect: e.g., the size of the effect was reduced when both formats used visualisations, short menu versions, and when the problem format matched the answer format for frequency versions.   

Conclusion(s): The meta-analysis supports the consensus that performance on Bayesian inference tasks is facilitated by natural frequency formats when compared to conditional probability formats.  Despite this result, a non-trivial amount of people continue to have difficulty with Bayesian inference problems even when presented as natural frequencies.  We discuss gaps in the literature, suggest future research directions, and suggest methodological approaches to capture further information about how people acquire and update information.  We advise how to improve the communication of medical test statistics to professionals and the public.

Purpose: The presence of uncertainty in the context of medical judgments and decision-making gives rise to the occurrence of cognitive biases and their detrimental effects on decision outcomes. Debiasing aims at reversing, eliminating, or reducing these negative effects. The purpose of the present study is to systematically review the existing research on debiasing in the medical context to systematize the field and identify opportunities and challenges for successful debiasing strategies.

Method(s): A systematic search of 14 electronic databases was complemented by hand search and resulted in 2143 abstracts eligible for screening. Of those, 55 articles reporting 67 relevant experiments tested the effectiveness of a debiasing strategy and thus met the predefined inclusion criteria. Two reviewers independently performed the screening procedure, data extraction, and quality appraisal using the QATSDD tool, whereby all inconsistencies were resolved through discussion.

Result(s): Of 55 reviewed articles, 58.2% (n=32) explicitly referred to “debiasing”. However, 25.0% (n=8) of these studies did not clarify the term’s meaning. Most experiments intended to debias optimism bias (n=24), followed by framing effects (n=10), and a biased statistical reasoning such as denominator neglect (n=13). Lay people, as opposed to health care professionals, were in 82.1% (n=55) of cases the target of debiasing efforts. Applying the categorization of Larrick (2004), the majority of studies employed a cognitive (n=30) or technological (n=21) debiasing strategy aiming at an alteration of participants’ way of thinking or the design of a more decision-friendly information environment, respectively. Methodological quality ranged from 31.0 to 92.9% (mean: 70.6%). The quality appraisal identified a lack of pilot-testing of experimental materials, insufficient reporting of sample size considerations, and the use of non-representative samples such as undergraduate students as main methodological limitations. Overall, 65.7% (n=44) of the debiasing strategies were found to be completely (n=27) or partially (n=17) successful.

Conclusion(s): In the past, debiasing was considered to be effortful and with only little prospects of success. Yet, the rise of novel technologies and the growing importance of informed decision-making and its accompanying tools such as decision aids seem to have sparked the new development of innovative debiasing strategies with a high success rate. Future debiasing studies could benefit from a stronger tie to the existing evidence-base and a consistent application of the underlying theoretical concepts including their terminology.

Background: A key policy approach in United States to curbing the growth health care costs is cost transparency.   To date, there has been limited research on consumers’ comprehension and use of public reports of health care cost and related efficiency measures despite research demonstrating that consumers find comparative quality data challenging. 

Purpose: We examine what type of cost data consumers prefer (out of pocket (OOP) or total costs, per visit or annual), what kind of symbol was effective for cost comparisons (descriptive icons, dollar amounts, or cost ratios), and how to effectively present readmission data, for which a lower score is better.

Method(s): We conducted three randomized online experiments with 448 U.S. employees of a large semiconductor company.  Participants viewed displays of comparative health provider information and were asked to hypothetically select a provider.  The same underlying information was presented; however, there were five different approaches to displaying the information.  In one experiment, for example, participants viewed comparative quality and cost information on four hospitals (high quality/low cost, high quality/expensive, and two mixed quality and cost), and the cost information varied in whether it was presented using descriptive icons (high cost, average, affordable; or high cost, average, low cost), with median costs in dollars, a combination of descriptive icons and median costs, or spending as a ratio compared to state median).

Result(s): Respondents were as interested in knowing their OOP costs per visit as they were in learning about consumer ratings of health providers.  Respondents, however, were significantly less (12%) interested in knowing about cost information when it was described as average annual total cost of care or average annual OOP costs.  When presented with comparative information on hospitals, respondents ranged in selecting the high value hospital 78% of the time, when cost was presented as a ratio, to 95% when a descriptive icon was used and low cost hospitals were labeled “affordable.”  In the readmissions experiment, 82% of respondents selected the highest quality hospital when descriptive icons (better, average, below) detailed readmissions performance rather only 56% when percentages were displayed.

Conclusion(s): This study confirms that consumers are interested in cost information, but presenting the information is tricky.  Displays should be tested with consumers prior to releasing comparative cost information to the public.