METHODS FOR STUDYING THE COGNITIVE CAUSES OF DIAGNOSTIC ERROR: A SYSTEMATIC REVIEW

Background: Cognitive factors are the most prevalent cause of diagnostic error. This systematic review aims to collate and characterize the methods that have been employed for their study.

Methods: MEDLINE, PubMed, EMBASE, PsycINFO and Web of Science were searched systematically to identify primary studies investigating the cognitive factors affecting the diagnostic performance of physicians. Studies of visual diagnostic tasks or mental health disorders were excluded.

Results: Searches identified 2742 studies of which 71 were eligible for the review. They were categorized into either 'experimental' (whereby situations for observing the behavior of interest are created by the researcher) or 'observational' (whereby situations for observing the behavior of interest are sampled from real cases). Experimental studies were subdivided into 'process-tracing' (that collect data during the diagnostic process) and 'post-hoc'. Observational studies were subdivided into 'record-based' (sampling error cases from databases) and 'clinician-based' (eliciting error cases from physicians).

Experimental (n =36)			Observational (n =35)
Process-tracing	Post-hoc	Overlapping	Record-based	Clinician-based	Overlapping	Insufficient Information for classification
20	15	1	26	5	2	2

Studies were characterized in terms of theoretical framework, potential for bias, generalisability and multiple methods. A theoretical framework to enable predictions and explain results was present in 87% of experimental studies and 14% of observational studies. Potential for bias: In experimental studies, reactivity (method could change the behavior being studied) was potentially a problem for 62% of process-tracing studies. In 71% of the post-hoc studies, bias stemmed from the use of post-hoc measures to make inferences about the diagnostic process. In a third of observational studies, there was subjectivity in the case review: a single or unspecified number of reviewers, lack of a review protocol and lack of measures of agreement. None of the clinician-based studies specified the time since the error occurred or whether the medical record was available during reporting, suggesting the potential for memory biases. Generalisability: Almost all experimental studies employed written simulations of patients. Each participant diagnosed on average 7 cases. Observational studies investigated a much larger number of diagnostic errors. However, there was substantial variation between studies in the number of records searched, cases analyzed and errors identified. Multiple methods: Only 8% of studies attempted to replicate or supplement findings through the use of multiple methods.

Conclusion: No single methodological approach is ideal for studying the cognitive causes of diagnostic error, yet multiple methodologies are rarely employed. Investigation into patterns of findings from each methodology is ongoing.