4J-3 AUTOMATING IDENTIFICATION OF MULTIPLE CHRONIC CONDITIONS IN CLINICAL PRACTICE GUIDELINE RECOMMENDATIONS

Tuesday, October 21, 2014: 4:00 PM

Tiffany Leung, MD, MPH1, Hawre Jalal, MD, MSc, PhD1, Donna Zulman, MD, MS1, Douglas K. Owens, MD, MS2 and Mary K. Goldstein, MD, MS3, (1)Department of Veterans Affairs, Menlo Park, CA, (2)Stanford University, Stanford, CA, (3)VA Palo Alto Health Care System and Stanford University, Palo Alto, CA
Purpose:

Two-thirds of Medicare beneficiaries have multiple chronic conditions (MCCs), or two or more chronic conditions, but clinical practice guidelines (CPGs) often provide recommendations for single conditions. We developed and evaluated an automated method to determine the extent to which disease-specific CPG recommendations mention comorbid conditions.

Method:

This study focuses on guidelines for the 15 most prevalent Medicare chronic disease diagnoses, excluding cancer given the breadth of the term, and adding obesity due to its high prevalence and clinical significance. We compiled a corpus of text from CPG summaries available in the National Guideline Clearinghouse. CPGs were included if only one of the 15 diseases was mentioned in the title and the target population was the general adult, non-pregnant population. Using disease synonyms obtained from ontologies accessible via BioPortal at the National Center for Biomedical Ontology, we developed a direct text matching algorithm to identify comorbid disease terms in the recommendation sections of included CPGs. We tabulated the proportion of CPGs mentioning comorbid disease terms, and the number of comorbid disease terms mentioned in each CPG. To evaluate the automated approach, manual annotation by two annotators with medical expertise performed sentence-level annotation on five randomly selected CPGs for different diseases to generate a preliminary reference standard.

Result:

We obtained 2,503 guideline summaries, and 243 met inclusion criteria. Guidelines for concordant diseases (diseases that are part of the same pathophysiologic risk profile), such as hypertension, diabetes, hyperlipidemia, and obesity, mentioned one another most frequently. Only hypertension was mentioned across all CPGs, while Alzheimer’s disease and osteoporosis were mentioned the least. Annotators agreed on 95.5% of 561 sentences in the reference standard. Compared to the reference standard, precision (or positive predictive value) of the automated method was 0.79, recall (or sensitivity, or true positive rate) was 0.86, and F-measure (the harmonic mean of precision and recall) was 0.83.

Conclusion:

We developed and evaluated an automated method that identifies comorbid disease terms in CPG recommendations. An annotation guide to improve the reference standard is in development and will guide algorithm improvements. This method may be useful to inform gaps in guideline recommendations regarding comorbid diseases and therefore identify opportunities for guideline improvement.  Further investigation is needed to understand the context and variation of comorbid disease mentions in CPGs.

See more of: HEART DISEASE: DECISION MAKING AND CLINICAL GUIDELINE IMPLEMENTATION

See more of: The 36th Annual Meeting of the Society for Medical Decision Making