BASE RATES AND EXTENSIONAL REASONING IN DECOMPENSATION RISK ASSESSMENTS: A LONGITUDINAL COHORT STUDY

Purpose: We conducted a longitudinal cohort study examining physician use of probabilities to assess patient risk of deterioration.

Method: Over a six month period, residents, interns, overnight interns, and attending physicians from the general internal medicine ward of a single academic tertiary care center were asked to assess: 1) the probability that each of their patients would experience a clinical deterioration in the next 24 hours (defined as a cardiac arrest, call to a rapid response team, or transfer to the ICU), and 2) the probability of just a rapid response team call for their last patient for the day. This latter probability was compared to the physician's assessed probability of the extensional (inclusive) event (cardiac arrest, rapid response team call, or transfer to the ICU).

Result: 135 unique physicians made 6,310 judgments across the entire study. Physicians remained in the study for either two or four weeks with four to eight data collection points. Judges were initially insensitive to the base rate (~2%) when predicting 24 hour decompensation risk for their patients (median = 10%). Over the duration of their rotation, their probability assessments significantly decreased (median = 5%) but remained consistently higher than the base rate. This pattern held for all cohorts of physicians (interns, residents, and attendings). Attendings violated the laws of probability by giving a lower probability to the extensional (inclusive) event 8% of the time, compared to 10% for residents, 12% for interns, and 19% for overnight interns. For the day teams, these violations tended to strongly cluster within physicians. For the overnight team, violations were more evenly distributed across physicians.

Conclusion: Physicians initially overestimated the probability of their patients deteriorating by a factor of 5, but over time gave probabilities about 2 times the base rate. The pattern held for cohorts of very different experience levels (interns, residents, attendings), suggesting that this learning was due to being explicitly asked to assess the probabilities. A minority of physicians consistently violated the extensional laws of probability. However, rates were higher (and more uniform) for overnight physicians, who may have been more cognitively taxed (due to sleep deprivation).