1A-5
IMPROVING UNDERSTANDING OF DIAGNOSTIC SCREENING TESTS THROUGH SIMULATED EXPERIENCE
Diagnostic screening tests require an understanding of conditional probabilities, specifically the positive predictive value—PPV—of a test (i.e., "given a positive test result, what is the probability that you have the disease?"). Evidence exists for a "collective statistical illiteracy", even amongst highly educated individuals (Gigerenzer et al., 2008). We investigated whether presenting statistical properties of screening tests by simulating test outcomes—simulated experience—improves understanding of conditional probabilities, compared to a descriptive presentation of statistical information.
Method:
Across two studies (Experiment 1, undergraduate students, N=64; Experiment 2, Internet sample recruited via Amazon Mechanical Turk, N=176), participants were presented with a vignette describing a person receiving a positive result from a full integrated screening test for Down syndrome. Participants were asked to estimate the PPV of the test. In the description condition, participants were provided the prevalence of Down syndrome and the sensitivity and specificity of the test—the PPV could be derived using Bayes' theorem. In the simulated experience condition, participants were presented with grids of 100 coloured squares denoting a representative sample of disease statuses and screening test results (see figure), blue representing false positives and orange representing true positives. Participants could sample up to 5000 fictitious test results and PPV could be estimated by tracking experienced outcomes. Experiment 2 also included three items measuring attitudes toward the test before and after providing PPV estimates.
Results:
Across the two experiments, participants' PPV estimates in the simulated experience condition were significantly more accurate than the description condition, (79% of the experience condition provided an estimate within 5 percentage points of the underlying PPV, compared with 14% of the description condition), p<.001. Experiment 2 revealed participants' attitudes towards screening significantly decreased after providing PPV estimates in the experience condition, but not the description condition, p<.05. Participants in the experience condition rated their likelihood of undergoing screening as 4.66 (6-point scale) 95%CI [4.37, 4.95] before and only 3.81 [3.47, 4.16] after estimating PPV. No differences were observed in the description condition.
Conclusions:
Simulated experiences can significantly improve PPV estimates while decreasing interest in screening. This intervention has the potential to dramatically improve patient understanding of diagnostic screening tests and may also be used to reduce reliance on questionable screening tests, and this reducing overtreatment.
