Candidate for the Lee B. Lusted Student Prize Competition
Purpose: Receiver operating characteristic (ROC) curves are commonly used to evaluate diagnostic tests, but many diseases have multiple risk factors or tests that could be used to perform these analyses. We compared ROC curves for 15 approaches (involving single or multiple risk factors or tests) of assessing cardiovascular disease (CVD) risk.
Method: We calculated 15 rankings of risk for 3,501 men and 2,498 women in the NHANES III population (baseline values 1988-1994) to compare ROC curves using 10-year CVD death as the outcome of interest. There were five categories of approaches evaluated: 1) Single risk factor (age, cholesterol, body-mass index [BMI], systolic blood pressure [SBP]); 2) Number (0-7) of dichotomous risk factors (age>55 years, LDL cholesterol>130 mg/dL, SBP>140 mmHg, BMI>30 kg/m2, diabetes, smoking, SBP treatment) with single risk factors as tiebreakers (age, cholesterol, BMI, SBP); 3) Total CVD risk (based on Framingham or non-laboratory-based risk scores); 4) Multistage (Framingham risk only available for 75%, 50% or 25% of population at intermediate risk, non-laboratory-based risk used for others); and 5) Combination of Framingham and non-laboratory-based risk (additive or multiplicative) for all individuals. Categories 1 and 2 relied on dichotomous and/or single risk factors, while Categories 3, 4 and 5 involved total risk scores. Categories 2, 4 and 5 consisted of multiple tests.
Result: In men, area under the ROC curve (AUC) results ranged from 0.474 (BMI single risk factor) to 0.782 (additive combination of Framingham and non-laboratory-based total risk scores). In women, this range was 0.556 (BMI single risk factor) to 0.834 (Framingham total risk score). All of the Category 1, 2, and 3 scores were statistically significantly worse (p<0.05) compared to the best score in each sex, except for age alone in men (AUC = 0.772), Category 2 tests with cholesterol or SBP as tiebreakers in women (AUCs of 0.807 and 0.827, respectively), and the non-laboratory-based total risk score in men (AUC = 0.782). AUCs for multistage tests ranged from 0.774-0.780 and 0.812-0.827 in men and women, respectively.
Conclusion: Tests involving total risk scores generally performed better than dichotomous and/or single risk factor-based tests. In men, age as a single risk factor performed comparably to the best scores (particularly at stricter positivity thresholds). In women, additional risk factor information beyond age significantly improved AUC results.