Purpose: Properly designed experiments have five advantages over observational studies: (1) Estimates are un-confounded (e.g. between race/ethnicity and socioeconomic status); (2) Cause and effect can be assessed directly; (3) They are cost efficient due to smaller sample size requirements; (4) They are free of selection biases (known and unknown); and (5) Observed covariates can be added to the analysis. Variations in medical decision making, such as discrimination by patient gender or race/ethnicity, are often examined using observational studies of massive data sets (e.g. claims data). However, results are confounded and compromised by selection biases and can only estimate possible associations. An experiment in which patient characteristics (but not medical details) are systematically varied allows unbiased cause and effect conclusions to be drawn with smaller samples.
Method: We used data from five factorial experiments concerning the diagnosis of coronary heart disease (CHD), depression, and diabetes. Between 256-384 randomly selected primary care physicians viewed video vignettes of patients. The patients varied by gender, age, race/ethnicity and socioeconomic status (depicted by dress and current or former occupation) with 16-24 different vignettes per experiment. Physicians were stratified by country, gender, and level of experience (measured by year of graduation from medical school). The patient characteristics were experimentally varied.
Result: We found that physicians were inconsistent in their attention to base rates of disease for various patients. Consistent with base rates, their certainty for a CHD diagnosis was lower for younger women (p <.01), and the probability of a diabetes diagnosis was lower in Whites compared to Blacks (p<.05). However, they were inconsistent with base rates in that their certainty of a depression diagnosis did not vary by gender and their probability of a diabetes diagnosis did not vary by age or socioeconomic status. When examining physician characteristics, we found (consistently across studies) that women would ask more questions. We also found that women and less experienced physicians would offer more lifestyle advice.
Conclusion: By experimentally manipulating patient characteristics we found significant non-medical (patient characteristics) influences on clinical decision making, with a relatively small number of physicians. We also found consistent physician effects. Our experimental design ensures these effects are un-confounded, free of selection biases and directly attributable to the manipulated causes.
Candidate for the Lee B. Lusted Student Prize Competition