PROVIDER EXPERIENCE USING A DECISION SUPPORT TOOL FOR PNEUMONIA: A MIXED METHODS STUDY

Purpose: We implemented a decision support tool to standardize management of pneumonia at four emergency departments (EDs). It led to a reduction in mortality [Dean, Ann. Emergency Med, 2013]. This real-time decision support tool provides recommendations for triage (disposition to hospital versus home) and antibiotic selection by integrating patient data with best practice guidelines, captures disagreements with each recommendation, and asks physicians why they disagreed. Our study's purpose was to examine provider experience using the tool by conducting a survey, measuring tool use, and analyzing patterns of disagreements with recommendations.

Method: We surveyed 100 physicians at 4 participating EDs using a 1-5 Likert scale regarding whether the tool was helpful for diagnosis, triage, and antibiotic selection. Among ED encounters in which the tool was used, we measured the proportion of those where the physician disagreed with the recommendation for triage and antibiotic selection. We used concept mapping to elucidate themes of the reasons physicians disagreed with tool recommendations: free text entries were parsed by 2 investigators, then sorted using an on-line card sorting exercise by10 clinicians. Structural distance matrix analysis generated a cluster tree.

Result: Providers (N=80) reported that the tool was most helpful for inpatient antibiotic recommendations (4.2), triage (3.6), and least with diagnosis (2.5). The tool was initiated on 1772 patient encounters, providing triage recommendations for 1772 and antibiotic recommendations for 1687. Physicians disagreed with 16% of all triage recommendations, compared to 6% of all antibiotic recommendations. Providers disagreed with recommendations to triage a patient home in 24% of cases and with recommendations to prescribe broad-spectrum antibiotics to inpatients in 10%. Reasons for disagreeing with triage clustered into: 1)alternative diagnoses, 2)additional signs of severity not measured by the tool, 3)failure of outpatient therapy, 4)functional status, and 5)comorbidities. Reasons for disagreeing with broad-spectrum antibiotics clustered into: 1)alternative diagnoses, 2)differences in pathogen risk assessment, 3)antibiotic allergies, and preferences of the admitting physician. (Figure)  When linking survey results to disagreements by provider, we found no correlation between survey ratings and agreement with recommendations.

Conclusion: Physicians found decision support for pneumonia helpful for antibiotic and triage decisions, although they often disagreed with its recommendations. Concept mapping of physician disagreements provides valuable feedback to enhance tool usability, identify barriers to guideline adherence, and improve understanding of decision-making.