First things first: risk model performance metrics should reflect the clinical application.

Area of Science:

• Biostatistics
• Epidemiology
• Medical Informatics

Background:

• Research into risk model performance metrics is ongoing, with criticism of the area under the ROC curve (AUC) for its perceived insensitivity to new biomarkers.
• Alternative metrics such as the two-category net reclassification index (NRI(p)), integrated discrimination index (IDI), and R-squared statistics have been proposed.

Purpose of the Study:

• To critically evaluate the utility of proposed alternative risk model performance metrics.
• To determine appropriate methods for assessing the predictive capacity of risk models, particularly in light of new biomarker integration.

Main Methods:

• The study reviews existing literature and critiques the application of alternative risk model performance metrics.
• It contrasts these metrics with decision-theoretic approaches for evaluating risk prediction models.

Main Results:

• The authors argue that the charge of insensitivity against the area under the ROC curve is not substantiated.
• Routine application of NRI(p), IDI, and R-squared statistics is not justified for all risk prediction models.
• Decision-theoretic measures are recommended when clinically relevant risk thresholds exist.

Conclusions:

• Performance metrics for risk prediction models should align with their intended clinical application.
• Decision-theoretic measures are preferred for models with established risk thresholds.
• Further research is necessary to develop suitable metrics for situations lacking defined risk thresholds.