A NATURALISTIC EVALUATION OF A DIAGNOSTIC SUPPORT SYSTEM FOR FAMILY PHYSICIANS

Purpose: To determine whether providing family physicians with a computerized diagnostic support system (DSS), integrated with the patient’s electronic health record (EHR), improves diagnostic accuracy. 

Method: A DSS prototype was designed and developed as part of the EU TRANSFoRm project (www.transformproject.eu). The prototype currently supports three reasons for encounter (RfE), abdominal pain, chest pain and dyspnoea, and is integrated with a commercial EHR system (InPS Vision3). It is triggered by the physician entering the RfE and immediately displays a list of suggested diagnoses for the specific patient, using also information extracted from the EHR. The principle of presenting family physicians with diagnoses to consider at the start of the encounter, before testing any diagnostic hypotheses, was shown to be effective with computer-simulated patients in two RCTs, in the UK and Greece (Kostopoulou and colleagues, 2015a & 2015b). In addition, the current prototype enables physicians easily to code both the presence and absence of symptoms and signs, while the list of suggested diagnoses is updated accordingly. At the end of the consultation, all the information that the physician has recorded is automatically transferred into the EHR.

In the evaluation study, 32 family physicians, users of the Vision3 EHR system, diagnosed 12 standardized patients (actors) in simulated clinics. Each physician first consulted with 6 patients using Vision3, and on a second occasion, with 6 different but matched for difficulty patients, using the DSS. The patient scenarios ranged in difficulty and were counterbalanced, so that they were all seen with and without the DSS across physicians. 

Result: Mean diagnostic accuracy was 0.50 [95% CI 0.42-0.58] without and 0.57 [0.50-0.64] with the DSS. Improvement in diagnostic accuracy was significant: odds ratio 1.33 [1.07-1.66] (P=0.01). The odds of giving a correct diagnosis doubled, on average, when scenario difficulty was accounted for in the regression model: OR 2.1 [1.43 to 2.83] (P<0.001).

Conclusion: This improvement in diagnostic accuracy is clinically significant, since the evaluation was done in a realistic environment, with actors as patients, and under the usual time pressures of the clinical consultation (10 minutes). Furthermore, this was the first time that the physicians were using the DSS prototype, following a 30-minute training session. Data analyses of the prototype’s usability and of patient satisfaction are on-going.