PURPOSE: To compare different ways to analyze diagnostic reasoning, we developed a method for analyzing decisions made in response to computer based vignettes according to both judgment analysis (multivariable linear models) and Bayesian analysis.  

   METHODS: We developed cases based on a previous analysis of the presence of pulmonary infiltrate in 1436 patients with cough illness. A new computer based interactive system allowed us to present case vignettes such that participants could select any or all of 16 tests and findings and respond to their presence or absence with revision of their probability of infiltrate in each case. We measured the probability revision for each cue, global probability, and order and frequency of cue selection. Participants, internal medicine house officers (n = 6) and third and fourth year medical students (n = 20), completed a series of 20 case vignettes. They were instructed to select cues in order of importance and to stop when further information would not change their decision. We compared relative cue weights for four specific approaches: a combination of order and frequency of cue selection, a Bayesian approach using likelihood ratios, judgment analysis, and self-explicated weights.  

   RESULTS: Each of the four methods resulted in different estimates of cue importance. The five cues with the highest relative weight from each method are reported as: relative weight (cue name) [Table 1].  

   Table 1. Standardized weights for top cues according to method.

Selection Order and Frequency	Bayesian (likelihood ratio)	Judgment Analysis	Self-explicated
22% (age)	14% (crackles)	13% (temperature)	8% (temperature)
12% (productive cough)	8% (temperature)	9% (crackles)	7% (history of fever)
10% (history of fever)	7% (rhonchi)	8% (productive cough)	7% (respirations)
10% (temperature)	7% (respirations)	7% (age)	7% (productive cough)
8% (dyspnea)	7% (productive cough)	7% (dullness)	7% (dullness)
38% (remaining cues)	57% (remaining cues)	55% (remaining cues)	64% (remaining cues)

   CONCLUSIONS: Although the top cues for the Bayesian and judgment analysis approaches were similar, average order and weighting differed considerably. The frequency and order of selection reflected the order in which cues were listed in the vignette and examination habits rather than diagnostic utility. These results suggest that diagnostic weighting of clinical factors is markedly influenced by the analytic method and will require further study.