PURPOSE: To determine whether "decision curve analysis", a novel method for evaluating prediction models, helps to establish the value of new biomarkers for prostate cancer screening.

METHODS: The value of a novel biomarker is often assessed using a measure of accuracy such as the area under the ROC curve (AUC). Decision analytic methods are more informative for clinical decision making, but typically require additional data on costs or utilities. We developed decision curve analysis (DCA), a simple method for evaluating prediction models that incorporates clinical consequences and requires only the dataset in which the models are validated. We applied our method to determine whether adding other biomarkers to prostate-specific antigen level (PSA) would improve biopsy decisions in men with elevated PSA. A key feature of DCA is the assumption that the threshold probability of disease at which a patient would opt for biopsy reflects his valuation of the relative harms associated with a false-positive prediction (discomfort and inconvenience, risk of infection) and a false-negative prediction (delayed cancer diagnosis). This theoretical relationship is then used to derive the clinical benefit of the model across different threshold probabilities: the data set is analyzed to determine the number of true- and false- positive predictions at each threshold (t) and the net clinical benefit is calculated as the true-positive rate minus false-positive rate multiplied by the odds of disease (t/(1-t)). Plotting clinical benefit against a range of threshold probabilities yields the decision curve. We analyzed 3 strategies: biopsy based on age and PSA; biopsy based on prediction model including age, PSA and other biomarkers; and "biopsy all". We assessed AUC and constructed decision curves for each strategy using data from 353 men with elevated PSA who underwent prostate biopsy.

RESULTS: A prediction model incorporating all biomarkers had a significantly higher AUC than the model including only age and PSA (0.783 vs. 0.715, p=0.003). However, application of DCA showed that use of the biomarkers, compared to biopsying irrespective of PSA and biomarker values, would only improve clinical outcome in the unusual case where a man demanded an unusually high probability of cancer (>50%) before he would agree to biopsy.

CONCLUSIONS: Decision curve analysis is a simple, novel method that provides readily applicable data on the clinical value of novel biomarkers.