Background: Cognitive factors are the most prevalent cause of diagnostic error. This systematic review aims to collate and characterize the methods that have been employed for their study.

Methods: MEDLINE, PubMed, EMBASE, PsycINFO and Web of Science were searched systematically to identify primary studies investigating the cognitive factors affecting the diagnostic performance of physicians. Studies of visual diagnostic tasks or mental health disorders were excluded.

Results: Searches identified 2742 studies of which 71 were eligible for the review. They were categorized into either 'experimental' (whereby situations for observing the behavior of interest are created by the researcher) or 'observational' (whereby situations for observing the behavior of interest are sampled from real cases). Experimental studies were subdivided into 'process-tracing' (that collect data during the diagnostic process) and 'post-hoc'. Observational studies were subdivided into 'record-based' (sampling error cases from databases) and 'clinician-based' (eliciting error cases from physicians).

Experimental (n =36)			Observational (n =35)
Process-tracing	Post-hoc	Overlapping	Record-based	Clinician-based	Overlapping	Insufficient Information for classification
20	15	1	26	5	2	2

Studies were characterized in terms of theoretical framework, potential for bias, generalisability and multiple methods. A theoretical framework to enable predictions and explain results was present in 87% of experimental studies and 14% of observational studies. Potential for bias: In experimental studies, reactivity (method could change the behavior being studied) was potentially a problem for 62% of process-tracing studies. In 71% of the post-hoc studies, bias stemmed from the use of post-hoc measures to make inferences about the diagnostic process. In a third of observational studies, there was subjectivity in the case review: a single or unspecified number of reviewers, lack of a review protocol and lack of measures of agreement. None of the clinician-based studies specified the time since the error occurred or whether the medical record was available during reporting, suggesting the potential for memory biases. Generalisability: Almost all experimental studies employed written simulations of patients. Each participant diagnosed on average 7 cases. Observational studies investigated a much larger number of diagnostic errors. However, there was substantial variation between studies in the number of records searched, cases analyzed and errors identified. Multiple methods: Only 8% of studies attempted to replicate or supplement findings through the use of multiple methods.

Conclusion: No single methodological approach is ideal for studying the cognitive causes of diagnostic error, yet multiple methodologies are rarely employed. Investigation into patterns of findings from each methodology is ongoing.

Background: This study tests the influence of different response modes (direct, after conscious and after unconscious thinking) in clinical decision making. Recently, we published a first demonstration of unconscious thought effects in this domain, specifically in the complex and error-prone task of diagnostic classification (De Vries et al., in press). The current study describes a follow-up and refinement, in three ways. First, it investigated the role of experience by including experienced clinicians. Second, it included the degree of difficulty of classifications. Third, it included a third (control) condition in which classifications were provided immediately after reading a case description.

Methods: We used two written case descriptions. Both cases represented co-morbidity, with a more familiar classification (low difficulty) and a much more unfamiliar classification (high difficulty). Participants were randomly assigned the task in two of three response modes: conscious-processing (i.e., deliberately thinking about the information in the case description), unconscious-processing (i.e., deciding after performing an unrelated distracter task), or direct responding, without any delay or intervening task. Our main dependent measure was the proportion of correct classifications.

Results: A GLM analysis revealed a significant three way interaction, see Figure 1. For classifications low in difficulty, novices performed best after conscious processing. Experienced clinicians, in contrast, performed worst after conscious processing. For classifications high in difficulty, experienced clinicians outperformed novices. Moreover, both conscious and unconscious processing resulted in better performance than immediate classification.

Figure 1: Proportion of correct classifications as a function of Processing Condition, Difficulty and Experience Level.

Conclusion: The occurrence of diagnostic errors in psychiatric classifications appears to be a complex function of information processing mode, level of experience and task difficulty. In case of familiar classifications, experienced diagnosticians perform best when they respond either directly, or after unconscious processing, probably due to a well-developed intuition through years of practice with similar cases. Conscious thought may make them take irrelevant information into account. Novices have not had many experiences with similar cases yet, but have recently learned explicit rules for classification tasks, which may explain why they perform best after conscious thought when a classification is relatively easy. When cases become more unfamiliar, no response is directly available in memory and further –conscious or unconscious-  information processing seems to be required. De Vries et al., (in press). The unconscious thought effect in clinical decision making: An example in diagnosis. Medical Decision Making.