Brief critical review: Statistical assessment of biomarker performance

Articles have suggested that C-statistics associated with receiver operator characteristic (ROC) curves are insufficiently sensitive compared to odds/risk ratios (OR/RR) defined by p-values. Some suggest reclassification techniques are more appropriate.

We review the concepts of p-values, OR/RR, and ROC curves. We construct a ROC curve, demonstrate parametric and nonparametric curves, and analyze a comparison of ROC curves.

Using these illustrations, we show that the ROC curve is not simply a C-statistic but a continuum of sensitivity/specificity pairs over decision levels. We demonstrate that p-values provide little useful information about discrimination and that OR/RR is a limited expression of a sensitivity/specificity plot. We illustrate that low prevalence produces low positive predictive values, reclassification techniques are subject to the same rules of prevalence as other analysis, and that modifying the analysis can decrease the p-value comparing C-statistics without altering the sensitivity/specificity plot.

ROC curves provide both visual and statistical information to support entry into large-scale trials, determining decision levels and the use of testing in the absence of such trials. If the sensitivity/specificity plot shows little improvement at appropriate decision point(s), statistically significant-improvement in other "novel" statistical indices should be suspect.