Purpose: There exist a number of risk models that predict the probability of diabetes-related complications and are often used by clinicians to guide and motivate patients in treatment decisions. Our objective is to evaluate the discrepancy in risk models for Coronary Heart Disease (CHD) and Stroke for diabetes patients. We evaluate three competing risk models from the literature: UKPDS, Archimedes, and Framingham. All three models predict the probability of CHD and Stroke.

Methods: Our evaluation used two approaches; i.e. 1) comparative statistics for the discrepancies between models including mean and variance of the absolute deviation for diabetes patients in 2001-2002 National Health and Nutrition Examination Survey (NHANES) and 2) formulation and solution to a Markov decision process (MDP) model for a particular decision related to treatment of diabetes patients, the choice of when to start statins. Our MDP assumes discrete 2-year decision epochs with a finite decision horizon between 40 and 80 years of age. At each year the patient may opt to begin statins. The model considers the cost of statins as well as the collective benefits of risk reductions for CHD and Stroke based on expected effects of statins on total cholesterol and HDL.

Results: We find that there are significant differences in predicted probabilities of complications between all three risk models. In particular, UKPDS and Archimedes have a mean absolute percentage error for 10 year risk of CHD of 9.1%, UKPDS and Framingham have a mean deviation of 8.7%, and Archimedes and Framingham have a mean deviation of 3.5%. Similarly, mean deviations for 10 year risk of stroke are 7.5%, 10.0%, and 4.2%, respectively. Through sensitivity analysis with our MDP model we find that the differences in risk can translate to as much as an 8 year difference in optimal age to start statin treatment between UKPDS and Archimedes, 10 year difference between UKPDS and Framingham, and a 4 year difference between Framingham and Archimedes.

Conclusions: Based on our analysis, we conclude that there are differences in alternative risk models to predict the probability of CHD and stroke among diabetes patients. Furthermore, such deviations translate into significant differences in the implied optimal time to start treatment with statins.