CHANGES IN STRUCTURE OF DIAGNOSIS KNOWLEDGE DUE TO A CHEST PAIN DIAGNOSIS TUTORIAL

Background: The KBIT tutorial approach presents many (>50) case examples of a target presentation, such as chest pain, to sharpen students’ ability to discriminate diagnostic categories. Its effects on the structure of domain knowledge per se have not been investigated. We measured changes in structure of knowledge, based on analysis of similarity judgments made before and after studying the KBIT tutorial, using the Pathfinder approach.

Methods:  Tutorial: Students studied 9 chest pain diagnoses’ symptom lists, then diagnosed 49 practice cases described in terms of history and physical, with multiple choice response and immediate error feedback. Accuracy of diagnoses and time spent were measured. Measurement of cognitive structure: Students judged similarities of selected pairs of items from the chest pain diagnosis domain:  diagnosis to diagnosis, diagnosis to finding, and finding to finding.  3 experts also provided judgments, and similarity statistics were derived from the disease-by-finding sensitivity matrix. The Pathfinder analysis develops a multidimensional representation of each student's and expert's knowledge, and measures difference between student and expert. Change from pre and post tutorial structures will reveal whether the students' structures become more similar to the experts'.

Results: 16 students (2 undergraduate, 7 2nd year, 4 3rd year, 3 4th year) completed the tutorial; only 8 of them did the similarity judgment task both pre- and post-tutorial, as directed.  The more education, the more accurate diagnoses on tutorial (r = .38, p = .15, N = 16) and the more time spent on correct (r = .44, p = .09) and incorrect (r=.42, p = .10) responses. Wrong answers took more time (.52) than right answers (.42, t = 2.7, p = .016). More education predicted pre-tutorial Pathfinder measures of knowledge more like the experts, but that was unrelated to tutorial accuracy. After the tutorial, the students had more Links (t = 2.44, p = .045, N = 8), a higher ratio of common links (with experts) to unique links (t = 2.58, p = .036), and marginally higher correlations between their judgments of similarity and the average of the experts' (t = 2.04, p = .08).

Conclusion: The effects of education prior to studying the tutorial are evident in the relation between educational level and accuracy on the tutorial's cases. Despite the small N, two of five Pathfinder measures indicated the students' knowledge structure became significantly more similar to the experts', after the tutorial.