Purpose: To examine how psychiatrists make judgements about acute-psychiatric in-patients suicide risk, including the level of consistency with which judgements are made between and within individuals. In addition, an exploration of the psychiatrists performance, based on both their ability to discriminate between cases, and their judgement consistency was also evaluated, based on the Cochran-Weiss-Shanteau (CWS) measure of expertise (1).

Methods: 12 psychiatrists, working in community mental health trusts in Scotland, UK, volunteered to take part in the study. They were given a total of 145 case vignettes (130 cases plus 15 repeat cases), representing patients with varying degrees of suicide risk. For each scenario participants were asked to say how likely it was that the patient would commit suicide within the next 24 hours, on a 10cm line from 0 (no risk) to 100 (very high risk).

Risk predictions from the 15 repeat cases were analysed for consistency using paired sample t-tests. The CWS was calculated using measures of both discrimination and inconsistency (1).

Results: The mean age of the participants was 39 years (SD 7.9; range 25-53), 50% were male, 50% female. The overall mean risk prediction for the 15 cases at time 1 was 56.53 (SD 11.8) and time 2, 52.6 (SD 12). A paired samples t-test indicated a significant difference in mean risk prediction scores for the psychiatrists between time 1 and time 2 (t=3.18; df =11; p = 0.009). CWS measures were calculated for each participant, with the highest ratio of 15.3, and the lowest 2.1. There was little agreement between psychiatrists on the risk judgements for each case (e.g. range for one case 2 – 75).

Conclusions: Psychiatrists were inconsistent in their judgements of suicide risk for the series of case vignettes used in this study. This inconsistency was both between psychiatrists (their risk predictions for the same patient cases varied significantly), and within individuals (they were inconsistent in their risk predictions for the same patient case when repeated). Psychiatrists performance varied with the majority neither showing discrimination or consistency. This study highlights the complexity of risk prediction in acute psychiatric environments and the need for examination of risk judgements in the clinical environment in more detail.

1. Weiss DJ. Shanteau J. Empirica Assessment of Expertise. Human Factors. 2003. 45(1): 104-116