Purpose: Evaluate a gist-based Visual Representation of Risk (VRR) tool that helps clinicians answer basic diagnostic questions about the risk of disease after diagnostic testing.

Method: Cased-based scenarios were presented to third-year medical students. Presented with the pre-test probability of coronary artery disease (CAD), as well as the sensitivity and specificity of the diagnostic test (exercise treadmill test), respondents were asked for the post-test probability given positive and negative test results. Cases with and without the VRR tool were compared. Two cases (one low and one high prior probability) were presented to each participant, with VRR randomly assigned to one of the two cases. Mean scores where obtained for estimates with positive and negative test results for each case and a two-tailed t-test was used to compare means. Data were collected before (N=77) and after (N=69 of the 77) the seminar.

Results: Before the seminar, and without the VRR, predictable errors occurred in post-test up-dating. For the low prior (5%), positive result: learners estimated 54%, but the correct response is 20%. For the high prior (85%), negative result: learners estimated 40%, but the correct response is 67%. Significant improvements with the VRR were obtained when test results were positive for both the low prior (54% vs. 33%; correct is 20%) and high prior (73% vs. 78%; correct is 96%). Estimates for negative test results trended in the right direction, but were non-significant.

After the seminar, all estimates improved and better reflected prior probabilities in accordance with Bayes' Theorem. The VRR tool improved performance still more, however, for cases with high priors for both positive and negative test results: 80% vs. 91% for a positive result when the correct answer is 96%; 51% vs. 62% for a negative result when the correct answer is 67%.

Conclusions: Deficits in post-test updating have been well described and interventions disappointing in overcoming these errors. A gist-based Visual Representation of Risk Tool significantly improved performance in the absence of instruction about Bayes' Theorem (before the seminar) and, for half of the cases, over and above the effects of instruction.