PURPOSE: Are doctors Bayesian or aren't they? Bayesian Communication (BC) integrates readers' elicited priors with study data and reports the implications specific to that reader. This study was designed to evaluate whether an online BC tool would elicit Bayesian behavior.

METHODS: An online trial of a custom Bayesian interface for the z-test designed through iterative design with academic clinicians. Subject were clinicians who read RCTs. The study randomized assignments by two factors, 2 treatments (moxifloxacin for pneumonia; omalizumab for asthma) and 2 sample sizes (50, 250), and exposed all subjects first to a frequentist-based (EBM) interface followed by a BC interface. Prior and posterior action preferences were assessed with the question, “To what extent would you consider using <Rx> to treat patients with <condition>?” In BC, the minimally clinically important difference (MCID) was elicited, along with prior belief in control-rate and in control–experimental-rate difference. “Bayesian behavior” was defined as (1) prior belief being sensitive to the domain, (2) prior action preference being sensitive to prior belief in the difference, (3) posterior action being driven by posterior probability and MCID, (4) posterior action being driven by data when sample size was larger.

RESULTS: 49 clinicians completed the trial, evenly distributed across professional age (0 to 20 years), genders, and having published in the past 2 years. (1) Prior distributions in the moxifloxacin scenario differed significantly from the those in the omalizumab (means of 70 (se 5.5) vs 18 (5.0), P < .0001), although priors on differences did not. (2) Prior action preference did not correlate with prior belief in difference. (3) Posterior action preference did not correlate with posterior probability in the difference, but all 9 cases where that probability was > .95, posterior action preference was greater than prior action preference. (4) There was no statistically significant effect of the BC interface inducing greater sensitivity to the sample size.

CONCLUSION: In this first controlled trial of BC in physicians with minimal training in Bayesian concepts, physicians displayed mixed Bayesian behavior. Adoption of BC will require greater attention to explaining the concepts and implications of the calculations.