
Method: Casedbased scenarios were presented to thirdyear medical students. Presented with the pretest probability of coronary artery disease (CAD), as well as the sensitivity and specificity of the diagnostic test (exercise treadmill test), respondents were asked for the posttest probability given positive and negative test results. Cases with and without the VRR tool were compared. Two cases (one low and one high prior probability) were presented to each participant, with VRR randomly assigned to one of the two cases. Mean scores where obtained for estimates with positive and negative test results for each case and a twotailed ttest was used to compare means. Data were collected before (N=77) and after (N=69 of the 77) the seminar.
Results: Before the seminar, and without the VRR, predictable errors occurred in posttest updating. For the low prior (5%), positive result: learners estimated 54%, but the correct response is 20%. For the high prior (85%), negative result: learners estimated 40%, but the correct response is 67%. Significant improvements with the VRR were obtained when test results were positive for both the low prior (54% vs. 33%; correct is 20%) and high prior (73% vs. 78%; correct is 96%). Estimates for negative test results trended in the right direction, but were nonsignificant.
After the seminar, all estimates improved and better reflected prior probabilities in accordance with Bayes' Theorem. The VRR tool improved performance still more, however, for cases with high priors for both positive and negative test results: 80% vs. 91% for a positive result when the correct answer is 96%; 51% vs. 62% for a negative result when the correct answer is 67%.
Conclusions: Deficits in posttest updating have been well described and interventions disappointing in overcoming these errors. A gistbased Visual Representation of Risk Tool significantly improved performance in the absence of instruction about Bayes' Theorem (before the seminar) and, for half of the cases, over and above the effects of instruction.
See more of Poster Session IV
See more of The 28th Annual Meeting of the Society for Medical Decision Making (October 1518, 2006)