AN EVALUATION OF BAYESIAN NETWORKS FOR THE REAL-TIME DETECTION OF ERRORS IN THE CLINICAL LABORATORY
Gregory B. Strylewicz, MS and Jason N. Doctor, PhD. University of Washington, Seattle, WA
Purpose: The purpose of this research is to evaluate the use of Bayesian networks against laboratory experts in the real-time identification of errors in clinical laboratory data. Methods: The authors used clinical laboratory data from 1,849 patients enrolled in a treatment of Type 2 diabetes randomized controlled trial and used a synthetic error generation process as the gold standard. Each patient's data set consisted of four laboratory analyses with low biological variability and three analyses with high biological variability along with historical results. Approximately 6% of the data sets contained erroneous laboratory results created by adding or subtracting a percentage of the original value. Approximately 5% had a 25% error and 1% had a 50% error. The structure of the Bayesian network was determined from the training data using 10-fold cross-validation and a greedy search algorithm. The authors used the trained Bayesian network on the test data set to estimate the probability that each laboratory test result was erroneous given the other measured values for that patient. Two laboratory experts whose normal duties include the review of laboratory results for error identification reviewed the test data set and annotated values they felt to be erroneous by clicking a button next to the value. Results: The experts evaluated an average of 185 sets and the Bayesian network evaluated 925 sets. Laboratory experts achieved an average sensitivity of 8.9% and specificity of 95.8% for errors in analytes with high biological variability and an average sensitivity of 57.7% and specificity of 87.9% for errors in analytes with low biological variability. The performance of the Bayesian networks was comparable or better than laboratory experts for all analytes. For example, at a comparable specificity the Bayesian network achieved a sensitivity of 83.7% compared to the expert's 53.8% for detecting errors in glycosylated hemoglobin. Conclusions: Both the Bayesian network and laboratory experts detected errors in analyses with low biological variability significantly better than errors in analyses with high biological variability. Both identified large errors better than small errors. The performance of Bayesian networks was comparable or better than laboratory experts, suggesting that Bayesian networks are an effective means for the real-time detection of errors in the clinical laboratory.