Method: We have developed a single page form to be filled in by the clinicians each time they make a decision of taking/not taking a thorax X-ray image of a child with suspected bronchiolitis. In the form, they fill in their decision, as well as clinical data like temperature, oxygen saturation, and general (health) condition. They also rate how important each of these clinical factors are in their decision-making process. Our decision theoretic framework is that of bounded rationality (Kahneman D. 2003: A perspective on judgement and choice, American psychologist 58, 9, 697-720).
Results: In order to work as regression weights, the importance ratings should be relatively constant, allowing for some variation due to disagreement between individual clinicians. However, we see strong correlations between variable values, importance ratings, and decisions, which clearly contradict an interpretation of importance ratings as regression weights. The tendency is that, in a given case, variables that support the actual conclusion are reported as important. This is particularly so for positive decisions: if the patient has a fever, and the clinician chooses to undertake X-ray, he reports temperature as important. If the general (health) condition is normal, he rates this fact as unimportant. Had it been the other way around, with reduced general (health) condition and little or no fever (but still positive X-ray decision), his importance ratings would likely switch.
Conclusion: In our study, clinicians rate the importance of different clinical variables with respect to their decisions of undertaking thorax X-ray imaging or not. They appear to use these importance weights as a means of explaining why they made their actual decision. Therefore, collected importance ratings cannot be interpreted as regression weights directly.