
Method: 24 resident physicians accessed an online diagnostic exercise and tool through Mayo Clinic's Enterprise Learning System from June 2005 to May 2006. Values for pretest probability, sensitivity, and specificity are entered by the physician user and the VRR is graphically displayed. Users may access the Bayesian equation for the task at any time in the process of answering the posttest diagnostic question. The online tool generates the question, “What is the probability of disease if the test is positive?” After a response is obtained feedback is given and the response is graded as correct or incorrect with an explanation shown using the VRR tool. A second question is asked, “What is the probability of disease if the test is negative?” A response is entered and immediate feedback provided. There is no limit for tool use. An online calculator is also provided.
Results: Multiple regression analyses were preformed using estimated pretest probability, sensitivity, specificity and whether equations were viewed as predictors. Dependent variables were analyzed separately and included differences between estimated and correct values for positive predictive value and for one minus negative predictive value. R2 were .49 for positive predictive value and .55 for one minus negative predictive value. Pretest probability was significant in both, as expected. Crucially, viewing Bayesian equations was associated with lower performance for the estimate that followed. A gistbased VRR tool significantly improved posttest updating performance, relative to historical performance.
Conclusions: Deficits in posttest updating have been well described and interventions disappointing in overcoming these errors. Physicians accessing an online VRR tool who accessed the verbatim posttest equation performed significantly worse compared to physicians who relied on the gistbased VRR tool alone (the latter improved performance). This result was predicted by fuzzytrace theory with important implications for decision tools and decision aids. Fuzzytrace theory is a robust explanatory framework that guides the development of online interactive tools in Mayo Clinic's Enterprise Learning System.
See more of Poster Session III
See more of The 28th Annual Meeting of the Society for Medical Decision Making (October 1518, 2006)