ORAL ABSTRACTS: RISK PERCEPTION AND COGNITIVE BIASES
Method(s): We recruited 1113 participants from a US national internet panel. They received different types of narratives about flu shots, in addition to a standard pamphlet from the Centers for Disease Control and Prevention (CDC). Participants were randomly assigned to a narrative that was (a) pro-vaccine or anti-vaccine, (b) presented by either a patient discussing their own experience, a physician discussing another patient’s experience, or a physician discussing the experience of 50 patients, and (c) presented before or after the CDC pamphlet. Pro-vaccine narratives described the flu experiences of patients who got the flu after not getting a flu shot, and anti-vaccine narratives described the flu experiences of patients who got the flu after getting a flu shot. Narratives were equivalent in length and wording. Participants subsequently indicated the probability that they would get vaccinated, perceptions of flu risk and severity with and without the flu shot, and their ratings of the quality of the presented narratives.
Result(s): Low-numerate individuals reported lower vaccination intentions, especially after reading anti-vaccine (vs. pro-vaccine) narratives. A multi-mediation analysis suggested that low-numerate individuals’ vaccination intentions were reduced by anti-vaccine narratives and boosted by pro-vaccine narratives, due to their perceiving narratives as relatively better than did high-numerate individuals. These findings held across other conditions, including whether narratives were provided by patients or physicians.
Conclusion(s): Health-care providers may add narrative information when presenting statistical evidence to inform patients’ decisions. As compared to high-numerate recipients, low-numerate recipients seem to rely more on such narrative information when making their decisions.
Method(s): Using an experiment with 1,490 consumers self-identified with diabetes, we compared participants who saw a context statement in a fictitious (but professionally developed) price comparison DTC drug ad with participants who did not. We also included a control group that saw a DTC ad without the price-comparison or context statements.
Result(s): A majority of participants (59% of those assigned to see it) did not accurately report seeing the context statement. This pattern emerged despite the context statement being presented prominently below the price claim in the top half of the ad, linked with an asterisk.
We also assessed perceptions of participants who did recognize the presence of the context statement. Among confirmed exposure participants, a one-way ANOVA revealed a main effect of condition on perceptions of whether Veridan (the fictitious drug advertised) and Lyrica (the actual drug to which Veridan was compared) are interchangeable, F(2, 899) = 9.57, p<.001. Planned contrasts revealed that consumers seeing the context statement were more likely to agree the ad communicated uncertainty regarding comparative risk and efficacy than consumers seeing the comparison without context statement (M =4.48 vs. 4.07, p<.002).
The context statement also noted price savings presented may not reflect actual savings by consumers or third-party payers. Among confirmed exposure participants, those seeing the context statement rated the price comparison as less accurate (M = 4.44, SD = 1.20) than those seeing the comparison without the context statement (M = 4.79, SD = 1.19), F(1, 472) = 9.351, p<.01.
Conclusion(s): When people read a context statement, they demonstrate intended uncertainty about risks, efficacy, and savings. Despite its prominence and placement, however, the majority did not notice the statement. Although results support the potential for developing comprehensible context statements to clarify price comparisons, consumer attention may limit their effectiveness.
Method(s): In five parallel-designed experiments, 975 participants from a general adult population (54.7% women, M = 35.0, SD=11.2 years old) were randomly allocated to one of five experiments. Each experiment followed a 2(format) × 4(scenario) mixed design. In each experiment, participants read the risk of contracting a disease during their trip abroad. The risk was presented either in a ‘1-in-X’ or ‘N-in-X*N’ format in four scenarios randomly presented: malaria, Ebola, flu, Lyme disease. Participants assessed the risk on either a verbal probability scale (Exp. 1), a numerical probability scale (0-100%, Exp. 2), an arbitrary frequency scale (X out of 286, Exp. 3), a numerical probability scale with a delayed presentation (Exp. 4) or an arbitrary frequency scale with a delayed presentation (Exp. 5). Participants also made decision whether to cancel the trip to respective countries.
Result(s): We replicated the ‘1-in-X’ effect when probability perception was measured with a verbal probability scale (Exp. 1, Hedges’ g = 0.63, 95% CI[0.35, 0.91]). In the remaining numerical scale experiments (Exp. 2-5), we found that both ratio formats led to probability over-estimation (on average by 5.2%, 95% CI[2.1%, 8.4%], estimated in a multilevel meta-analysis). The ‘1-in-X’ formats triggered consistently higher subjective probability than ‘N-in-X*N’ formats: multi-level meta-analytical effect was g = 0.18, 95% CI[0.05, 0.32]. The ‘1-in-X’ ratio formats affected participants’ decision-making as they led to a higher willingness to cancel the trip abroad, aggregated effect across scenarios in Exp. 1-5, g= 0.17, 95% CI[0.04, 0.29].
Conclusion(s): Participants overestimated actual objective probabilities in both ratio formats. Since the ‘1-in-X’ effect was observed in all numerical scales (Exp. 2-5), bigger overestimation occurred in the ‘1-in-X’ format conditions. Health professionals should use ‘1-in-X’ formats with caution, because they make medical probabilities look bigger than they really are and, in turn, affect related decision-making.
Method(s): To identify papers that compared conditional probability and natural frequency Bayesian reasoning tasks, we conducted a systematic review across three major scientific and medical databases: Ovid(psych), Web of Science, and Pubmed. Cited reference searches were conducted on key papers and we requested relevant or unpublished papers from the JDM mailing list that we may have otherwise missed in our systematic search. Thirty relevant papers were identified from which 90 effects were subsequently analysed.
Result(s): A broad range of potential moderators were coded. These included moderators related to individual characteristics (e.g., numeracy, education), problem representation (e.g., congruence between answer and problem format, use of visual aids, menu), and methodology (e.g., use of incentives, scoring protocol). Results revealed the expected natural frequency facilitation effect: on average, performance was enhanced for natural frequency formats when compared to conditional probabilities. A number of variables moderated the effect: e.g., the size of the effect was reduced when both formats used visualisations, short menu versions, and when the problem format matched the answer format for frequency versions.
Conclusion(s): The meta-analysis supports the consensus that performance on Bayesian inference tasks is facilitated by natural frequency formats when compared to conditional probability formats. Despite this result, a non-trivial amount of people continue to have difficulty with Bayesian inference problems even when presented as natural frequencies. We discuss gaps in the literature, suggest future research directions, and suggest methodological approaches to capture further information about how people acquire and update information. We advise how to improve the communication of medical test statistics to professionals and the public.
Method(s): A systematic search of 14 electronic databases was complemented by hand search and resulted in 2143 abstracts eligible for screening. Of those, 55 articles reporting 67 relevant experiments tested the effectiveness of a debiasing strategy and thus met the predefined inclusion criteria. Two reviewers independently performed the screening procedure, data extraction, and quality appraisal using the QATSDD tool, whereby all inconsistencies were resolved through discussion.
Result(s): Of 55 reviewed articles, 58.2% (n=32) explicitly referred to “debiasing”. However, 25.0% (n=8) of these studies did not clarify the term’s meaning. Most experiments intended to debias optimism bias (n=24), followed by framing effects (n=10), and a biased statistical reasoning such as denominator neglect (n=13). Lay people, as opposed to health care professionals, were in 82.1% (n=55) of cases the target of debiasing efforts. Applying the categorization of Larrick (2004), the majority of studies employed a cognitive (n=30) or technological (n=21) debiasing strategy aiming at an alteration of participants’ way of thinking or the design of a more decision-friendly information environment, respectively. Methodological quality ranged from 31.0 to 92.9% (mean: 70.6%). The quality appraisal identified a lack of pilot-testing of experimental materials, insufficient reporting of sample size considerations, and the use of non-representative samples such as undergraduate students as main methodological limitations. Overall, 65.7% (n=44) of the debiasing strategies were found to be completely (n=27) or partially (n=17) successful.
Conclusion(s): In the past, debiasing was considered to be effortful and with only little prospects of success. Yet, the rise of novel technologies and the growing importance of informed decision-making and its accompanying tools such as decision aids seem to have sparked the new development of innovative debiasing strategies with a high success rate. Future debiasing studies could benefit from a stronger tie to the existing evidence-base and a consistent application of the underlying theoretical concepts including their terminology.
Purpose: We examine what type of cost data consumers prefer (out of pocket (OOP) or total costs, per visit or annual), what kind of symbol was effective for cost comparisons (descriptive icons, dollar amounts, or cost ratios), and how to effectively present readmission data, for which a lower score is better.
Method(s): We conducted three randomized online experiments with 448 U.S. employees of a large semiconductor company. Participants viewed displays of comparative health provider information and were asked to hypothetically select a provider. The same underlying information was presented; however, there were five different approaches to displaying the information. In one experiment, for example, participants viewed comparative quality and cost information on four hospitals (high quality/low cost, high quality/expensive, and two mixed quality and cost), and the cost information varied in whether it was presented using descriptive icons (high cost, average, affordable; or high cost, average, low cost), with median costs in dollars, a combination of descriptive icons and median costs, or spending as a ratio compared to state median).
Result(s): Respondents were as interested in knowing their OOP costs per visit as they were in learning about consumer ratings of health providers. Respondents, however, were significantly less (12%) interested in knowing about cost information when it was described as average annual total cost of care or average annual OOP costs. When presented with comparative information on hospitals, respondents ranged in selecting the high value hospital 78% of the time, when cost was presented as a ratio, to 95% when a descriptive icon was used and low cost hospitals were labeled “affordable.” In the readmissions experiment, 82% of respondents selected the highest quality hospital when descriptive icons (better, average, below) detailed readmissions performance rather only 56% when percentages were displayed.
Conclusion(s): This study confirms that consumers are interested in cost information, but presenting the information is tricky. Displays should be tested with consumers prior to releasing comparative cost information to the public.