EXTRACTING OPERATING CHARACTERISTICS THAT ARE “BAKED INTO” DISEASE ASSOCIATION MEASURES

Purpose: Diagnostic test performance is typically assessed using operating characteristics (i.e., sensitivity and specificity), but disease association measures (such as relative risk [RR]) are often reported instead. We sought to derive formulas to solve for operating characteristics from such papers containing the disease association measure, prior risk (or prevalence) of disease, and probability of positive test.

Method:

We used Bayes Theorem and other known formulas to derive the following equations:

1. Prior = p(D+) = PPV*p(T+) + (1-NPV)*[p(T-)]

2. RR = PPV/(1-NPV)

3. From 1) and 2): 1-NPV = p(D+) / [(RR-1)*p(T+) + 1]

4. From 2) and 3): PPV = RR*p(D+) / [(RR-1)*p(T+) + 1]

5. From Bayes: p(T+, D+) = p(D+|T+)*p(T+) = PPV*p(T+)

6. From Bayes: p(T-, D-) = p(D-|T-)*p(T-) = NPV*p(T-)

7. Sensitivity = p(T+|D+)

8. Specificity = p(T-|D-)

9. From Bayes and 7): Sensitivity = p(T+, D+)/p(D+)

10. From 5) and 9): Sensitivity = PPV*p(T+)/p(D+)

• p(T+) and p(D+) are givens; PPV is derived from 4)

11. From Bayes and 8): Specificity = p(T-, D-)/p(D-)

12. From 6) and 11): Specificity = NPV*p(T-)/p(D-)

• p(T-) and p(D-) are givens; NPV is derived from 3)

Equations 10) and 12) show how the operating characteristics can be calculated from probabilities that are typically known from papers that report disease association measures. We applied these equations to cerebrovascular reserve (CVR) and intraplaque hemorrhage (IPH) imaging, two separate markers of stroke risk in carotid stenosis patients. Recently published meta-analyses for each of these tests report associations in terms of odds ratios (approximations of RR) for stroke.

Result: The known annual risk of stroke in this population is 1.13%. The reported odds ratio of stroke for a positive CVR test is 3.86 with a probability of CVR test positive of 39%. For IPH, the odds ratio is 4.59 with a probability of test positive of 50%. The back-calculated sensitivity and specificity values were 0.71 and 0.61 for CVR, and 0.82 and 0.50 for IPH, respectively.

Conclusion: Researchers can now derive operating characteristics from reported disease association measures if they also use the known prior and dichotomous test result probabilities from such reports.  Doing so can provide test performance information needed for decision analyses.