Graphical representations and summary indicators to assess the performance of risk predictors

The availability of novel biomarkers in several branches of medicine opens room for refining prognosis by adding factors on top of those having an established role. It is accepted that the impact of novel factors should not rely solely on regression coefficients and their significance but also on predictive power measures, such as Brier score and ROC-based quantities. However, novel factors that are promising at the exploratory stage often result in disappointingly low impact in the predictive power. This motivated the proposal of the net reclassification improvement and the integrated discrimination improvement, as direct measures of predictive power gain due to additional factors based on the concept of reclassification tables. These measures became extremely popular in cardiovascular disease and cancer applications, given the apparently easy interpretation. However, recent contributions in the biostatistical literature enlightened the tendency to indicate as advantageous models obtained by adding unrelated factors. These measures should not be used in practice. A further measure proposed a decade ago, the net benefit, is becoming a standard in assessing the consequences in terms of costs and benefits when using a risk predictor in practice for classification. This work reviews the conceptual formulations and interpretations of the available graphical methods and summary measures for evaluating risk predictor models. The aim is to provide guidance in the evaluation process that from the model development brings the risk predictor to be used in clinical practice for binary decision rules.