Colleen Vrbin, BS, University of Pittsburgh, Pittsburgh, PA and Cleotilde Gonzalez, PhD, Carnegie Mellon University, Pittsburgh, PA.
Purpose: To determine the performance effect of presenting ambiguous versus unambiguous probabilistic associations between symptoms and diseases in a dynamic simulated medical diagnosis task. Methods: MEDIC, presented at last year's SMDM annual meeting, simulates a medical decision task in which subjects can run up to four tests to identify symptoms, make diagnoses, and provide a treatment to patients suffering from one of four diseases of equal base rate. Forty subjects were equally divided into two conditions. The first condition (P1) employed a probability matrix with unambiguous values of 0 and 1; perfect associations between symptoms and diseases. The second condition (P2) utilized a probability matrix with ambiguous values of 0.1, 0.5 and 0.9; stronger or weaker associations between symptoms and diseases. Performance in this experiment was measured by summing scores of three equally weighted components: patient health, diagnostic probabilistic accuracy and treatment effectiveness. The health component measured the amount of the patient's life preserved throughout the diagnostic process, with the patient's health fluctuating while awaiting test results. The diagnostic probabilistic accuracy component measured the difference between the subject's rated probabilities and the actual probabilities for each disease. The treatment effectiveness score was standard, based on effective treatment with either correct diagnosis (full points) or incorrect diagnosis (partial points), and ineffective treatment (no points). Results: Performance by subjects in P1 was statistically significantly higher than P2, with the average overall score per trial of 86.7 versus 76.1 points (p<0.001). Average health score, diagnostic probabilistic accuracy score and treatment effectiveness score per trial for subjects in P1 and P2 was 28.2 versus 27.4 points (p<0.001), 27.9 versus 23.6 points (p<0.001), and 30.6 versus 25.1 points (p<0.001), respectively. Subjects in P1 spent an average of 51 seconds per trial as opposed to 71 seconds per trial by the subjects in P2, (p<0.001). Conclusion: Unambiguous associations between symptoms and diseases led to significantly higher performance and 28% less time per patient. These results suggest that ambiguous associations between symptoms and diseases caused subjects to take more time to choose a diagnosis and treatment with no performance benefits. Future research is needed to improve probability learning in the inherently ambiguous field of diagnostic medicine to make diagnosticians more accurate and efficient in selecting optimal testing and treatment strategies.